Surface Feature Extraction Research Articles

PointSurFace: Discriminative point cloud surface feature extraction for 3D face recognition

Due to the geometric information in the 3D face data, the 3D face recognition methods exhibit better robustness against the physical attacks compared to the 2D recognition methods. In this paper, we propose PointSurFace, a 3D face point cloud recognition method integrated with a specially designed facial surface feature extraction (SurF) module. In PointSurFace, SurF comprehensively combines the coordinate positions, normal vectors, and angle relationships, etc., of each patch in the 3D face point cloud, to explore the explicit local geometric information of 3D faces and generate facial surface features. Subsequently, the surface features are fed into an encoder consisting of four set abstraction (SA) modules and three Inverted Residual MLP (InvResMLP) modules to extract more discriminative 3D facial features, where InvResMLP could suppress the overfitting and reduce information loss in the model training. In addition, PointSurFace utilizes channel fusion operations to integrate position information into the extracted 3D facial features, further enhancing the model performance. Experimental results in 3D facial recognition and verification across multiple datasets demonstrate that PointSurFace achieves the best 3D face recognition performance to date. For example, on Lock3DFace, it achieves the state-of-the-art recognition and verification accuracy of 90.03% and 80.01%, respectively, surpassing the previous methods by 2.02% and 3.19%. The ablation studies evaluate each module of PointSurFace, suggesting that the proposed surface feature extraction module significantly enhances the model performance.

MRFA-Net: Multi-Scale Receptive Feature Aggregation Network for Cloud and Shadow Detection

The effective segmentation of clouds and cloud shadows is crucial for surface feature extraction, climate monitoring, and atmospheric correction, but it remains a critical challenge in remote sensing image processing. Cloud features are intricate, with varied distributions and unclear boundaries, making accurate extraction difficult, with only a few networks addressing this challenge. To tackle these issues, we introduce a multi-scale receptive field aggregation network (MRFA-Net). The MRFA-Net comprises an MRFA-Encoder and MRFA-Decoder. Within the encoder, the net includes the asymmetric feature extractor module (AFEM) and multi-scale attention, which capture diverse local features and enhance contextual semantic understanding, respectively. The MRFA-Decoder includes the multi-path decoder module (MDM) for blending features and the global feature refinement module (GFRM) for optimizing information via learnable matrix decomposition. Experimental results demonstrate that our model excelled in generalization and segmentation performance when addressing various complex backgrounds and different category detections, exhibiting advantages in terms of parameter efficiency and computational complexity, with the MRFA-Net achieving a mean intersection over union (MIoU) of 94.12% on our custom Cloud and Shadow dataset, and 87.54% on the open-source HRC_WHU dataset, outperforming other models by at least 0.53% and 0.62%. The proposed model demonstrates applicability in practical scenarios where features are difficult to distinguish.

Image Super-Resolution Reconstruction Model Based on Multi-Feature Fusion

Due to the limitations of imaging equipment and image transmission conditions on daily image acquisition, the images acquired are usually low-resolution images, and it will cost a lot of time and economic costs to increase image resolution by upgrading hardware equipment. In this paper, we propose an image super-resolution reconstruction algorithm based on spatio-temporal-dependent residual network MSRN, which fuses multiple features. The algorithm uses the surface feature extraction module to extract the input features of the image, and then uses the deep residual aggregation module to adaptively learn the deep features, and then fuses multiple features and learns the global residual. Finally, the high-resolution image is obtained through the up-sampling module and the reconstruction module. In the model structure, different convolution kernels and jump connections are used to extract more high-frequency information, and spatio-temporal attention mechanism is introduced to focus on more image details. The experimental results show that compared with SRGAN, VDSR and Laplacian Pyramid SRN, the proposed algorithm finally achieves better reconstruction effect, and the image texture details are clearer under different scaling factors. In objective evaluation, the peak signal-to-noise ratio (PSNR) and structure similarity (SSIM) of the proposed algorithm are improved compared with SRGAN.

Organ-Level Instance Segmentation of Oilseed Rape at Seedling Stage Based on 3D Point Cloud

Highlights Provides a low-cost, fast, accurate, and non-destructive method for segmenting stems and leaves of oilseed rape at the seedling stage. Different varieties and leaf shapes of oilseed rape can all be accurately and efficiently segmented at organ level. The results of accurate organ instance segmentation are used for further organ morphological structure analysis and organ-level phenotype extraction. The calculated phenotypic data is highly correlated with the true phenotypic data. Abstract. Organ-level plant point cloud instance segmentation is crucial for three-dimensional (3D) plant structure investigations and plant phenotypic formations. Due to the complexity of 3D plant structures, accurate plant organ segmentation methods remain a bottleneck in current development of plant organ phenotypes. In this work, we present a fast, accurate, low-cost, and non-destructive method of stem and leaf instance segmentation at the organ level for oilseed rape point cloud. Firstly, we use a 3D scanner to obtain the point cloud of oilseed rape plants. Then, principal component analysis (PCA) is used to perform surface feature extraction on the point cloud after preprocessing. Subsequently, the stems and leaves in different features are segmented by region growing. The irrational segmentation results are finally optimized. Segmentation results of stems and leaves of oilseed rape of different sizes and leaf shapes are evaluated using manually segmented oilseed rape point clouds as a benchmark. The precision, recall, F-score, and average accuracy of the proposed stem and leave segmentation method are 0.983, 0.897, 0.937, and 0.883, respectively. The experimental results suggest that the proposed method can accurately segment an oilseed rape 3D point cloud into instances corresponding to the organs of the plant. Additionally, the separated leaf and stem organs of oilseed rape can be further used for plant structure studies and organ phenotype extraction. The organ phenotypic parameters of oilseed rape are calculated based on the results of stem and leaf instances segmentation. The leaf area, stem length and stem angle of oilseed rape calculated from the point cloud are also compared with the corresponding manual measurements, which are highly correlated with coefficients of determination R2 (0.82-0.97). In conclusion, the proposed segmentation method can be applied as a fundamental segmentation step to extract organ phenotype data from oilseed rape plants. Namely, our research is valuable for precision breeding and basic plant research. Keywords: 3D point cloud, Oilseed rape plant, Phenotypic traits, Stem-leaf segmentation.

Control technologies of lower limb rehabilitation exoskeleton robots based on surface electromyography: A review

The aging population is accompanied by a decline in human body function, leading to an increasing number of people with lower limb dysfunction, which has become a global public health challenge today. The lower limb rehabilitation exoskeleton robot based on surface electromyography is a current research hotspot. It can help people with lower extremity dysfunction perform better rehabilitation training. This review presents the analysis and processing of surface electromyography, feature extraction and recognition, as well as the control methods for lower limb rehabilitation exoskeleton robots.

A-SATMVSNet: An attention-aware multi-view stereo matching network based on satellite imagery

Introduction: The stereo matching technology of satellite imagery is an important way to reconstruct real world. Most stereo matching technologies for satellite imagery are based on depth learning. However, the existing depth learning based methods have the problems of holes and matching errors in stereo matching tasks.Methods: In order to improve the effect of satellite image stereo matching results, we propose a satellite image stereo matching network based on attention mechanism (A-SATMVSNet). To solve the problem of insufficient extraction of surface features, a new feature extraction module based on triple dilated convolution with attention module is proposed, which solves the problem of matching holes caused by insufficient extraction of surface features. At the same time, compared with the traditional weighted average method, we design a novel cost-volume method that integrates attention mechanism to reduce the impact of matching errors to improve the accuracy of matching.Results and discussion: Experiments on public multi-view stereo matching dataset based on satellite imagery demonstrate that the proposed method significantly improves the accuracy and outperforms various previous methods. Our source code is available at https://github.com/MVSer/A-SATMVSNet.

Surface-GCN: Learning interaction experience for organ segmentation in 3D medical images.

Accurate segmentation of organs has a great significance for clinical diagnosis, but it is still hard work due to the obscure imaging boundaries caused by tissue adhesion on medical images. Based on the image continuity in medical image volumes, segmentation on these slices could be inferred from adjacent slices with a clear organ boundary. Radiologists can delineate a clear organ boundary by observing adjacent slices. Inspired by the radiologists' delineating procedure, we design an organ segmentation model based on boundary information of adjacent slices and a human-machine interactive learning strategy to introduce clinicalexperience. We propose an interactive organ segmentation method for medical image volume based on Graph Convolution Network (GCN) called Surface-GCN. First, we propose a Surface Feature Extraction Network (SFE-Net) to capture surface features of a target organ, and supervise it by a Mini-batch Adaptive Surface Matching (MBASM) module. Then, to predict organ boundaries precisely, we design an automatic segmentation module based on a Surface Convolution Unit (SCU), which propagates information on organ surfaces to refine the generated boundaries. In addition, an interactive segmentation module is proposed to learn radiologists' experience of interactive corrections on organ surfaces to reduce interactionclicks. We evaluate the proposed method on one prostate MR image dataset and two abdominal multi-organ CT datasets. The experimental results show that our method outperforms other state-of-the-artmethods. For prostate segmentation, the proposed method conducts a DSC score of 94.49% on PROMISE12 test dataset. For abdominal multi-organ segmentation, the proposed method achieves DSC scores of 95, 91, 95, and 88% for the left kidney, gallbladder, spleen, and esophagus, respectively. For interactive segmentation, the proposed method reduces 5-10 interaction clicks to reach the same accuracy. To overcome the medical organ segmentation challenge, we propose a Graph Convolutional Network called Surface-GCN by imitating radiologist interactions and learning clinical experience. On single and multiple organ segmentation tasks, the proposed method could obtain more accurate segmentation boundaries compared with other state-of-the-artmethods.

ESTIMATION OF LANDCOVER TYPES OVER HIMALAYAN REGION WITH THE CLASSIFICATION OF OPTICAL AND MICROWAVE-BASED IMAGE FUSION DATASET

Abstract. Himalayas play a significant role in terms of climate influence, the origin of rivers, hydropower generation, tourism, and forest wealth. The monitoring of the rugged terrain Himalayas via remote sensing is one of the efficient solutions to meet future requirements. In remote sensing, the sensors can be categorized as optical and microwave. The optical-based sensor provides multispectral or hyperspectral information at a very fine spatial resolution but is limited to daytime images without any penetration through the clouds. Whereas, the microwave works more effectively due to day/night image acquisition and cloud penetration capabilities. Therefore, the image fusion of multi-sensors (optical and microwave) datasets is important to extract crucial information about the Earth surface, especially over the Himalayas. However, the main aim of the article is to retrieve the different landcover types using various classifiers i.e., Linear Spectral Mixing (LSM), Random Forest Classifier (RFC), and Support Vector Machine (SVM) on the fused dataset. The dataset has been acquired over a part of Indian Himalayan terrain i.e., Uttarakhand State, India using microwave-based ISRO’s Scatterometer Satellite (SCATSAT-1) and optical-based NASA’s Moderate Resolution Imaging Spectroradiometer (MODIS). The results show the effectiveness of the RFC classifier in the mapping of land surface features as compared to other classification algorithms i.e., LSM and SVM. This study not only highlights the potential of the RFC classifier in the extraction of information but also, shows the significance of fusion of optical and microwave datasets in the extraction of important Earth surface features.

Tactile Rendering of Textile Materials

Tactile rendering attempts to replicate touch stimuli with the aid of haptic devices so that the perception of a material can be stored and transmitted to a remote user. Haptic devices have potential implications on prosthetics, rehabilitation, telemedicine, and virtual reality. Research on haptic interfaces and novel approaches to rendering tactile feedback in virtual and immersive environments has grown recently. We conducted a comprehensive analysis to obtain information on tactile rendering, in general, from scholarly works published between 2009 and 2019. We present a consolidated report on the current techniques on tactile rendering of textile materials. Existing research on tactile rendering focused on three significant areas: surface feature extraction, virtual texture simulation, and assessment by a human observer. We found that in the majority of the cases, the tactile interface system uses a mechatronics framework, where one or more electromechanical transducers converted tiny electrical signals into mechanical forces to generate the necessary haptic stimuli. Furthermore, the paper discusses the prevailing critical challenges in tactile rendering and the directions for future research.

Facial bones segmentation from skull point clouds based on deviation angle

AbstractFacial bones segmentation is an important step to understanding a skull. In this paper, a method for segmenting facial bones from skull point clouds is proposed. The segmentation is based on the deviation angle features. The method consists of three phases: surface normal estimation, feature extraction, and point clouds classification. The method is applied to skull point clouds derived from computed tomography images. For evaluation, the method is compared with manual segmentation. The method has succeeded in segmenting facial bones with Precision = 0.836, Recall = 0.951, and F = 0.890.

Multidimensional spatial clustering and visualization of 3D topographic relief data

Point cloud contains a large collection of point data as surface returns to describe the multidimensional aspect of the topographic features. Data points within these point clouds at the atomic level convey little or no information about the structural, physical, and spatial information about objects of interest within the scene in its raw form. Analyzing the surface described as a set of the point is complex and challenging. This paper describes the framework and methods for analyzing topographical reliefs and terrain regions to extract surface features using spatial clustering and aggregation of point cloud data. The method also describes the effectiveness of Kd- tree and nearest neighbor estimates in segmenting and visualizing the surface topographical terrain and relief region. The morphological correspondence between the points within clusters and the corresponding surface is demonstrated. It is also defined as clusters and spatially varying aspects of the object structures for further studies. The result obtained with the experiments shows significant improvement in classifying and clustering small and large surface structures of topographic reliefs over other known methods. It establishes the relevance of using spatial clustering and the use of surface normals for clustering object structures of varying sizes on the earth's surface.

一种适用于行星表面特征提取的实时SIFT算法

一种适用于行星表面特征提取的实时SIFT算法

Performance testing of selected spectral indices in automated extraction of impervious built-up surface features using Resourcesat LISS-III image

The increasing urban density and spatial expansion of urbanized areas result in changes from the natural landscape to impervious surface features. Remote sensing–based spectral indices provide an efficient method in the automated identification of land use and cover classes. However, a common challenge is the accurate extraction of built-up features from satellite images. The original Normalized Difference Built-up Index (NDBI) has been modified by several researchers in the anticipation of improvement of the built-up area classification. The indices adopted in the study are Index-based Built-up Index (IBI), Built Up Index (BUI), NDBI, Modified Built-up Index (MBI), and the newly developed Impervious Built-up Index (IBUI). These indices work on automated kernel-based probabilistic thresholding algorithm to group the index values into built-up and non-built-up areas. This study investigates the performance of the above mentioned five spectral indices on Resourcesat LISS III imageries over the Kolkata-Howrah urban agglomeration, India. The indices were compared and found based on the overall accuracy that the novel IBUI provided the maximum accuracy (94%) in extracting impervious built-up areas. Based on the overall accuracy and kappa statistic the IBUI was chosen for estimating spatio-temporal urban growth of the study area from 2003 to 2018 based on the same LISS-III sensor data. It was calculated that the urban built-up area has increased from 144 to 177 km2.

Probabilistic Neural Network with Bayesian-based, spectral torque imaging and Deep Convolutional Autoencoder for PDC bit wear monitoring

Drill bit wear monitoring plays an important role during drilling operations, particularly in drilling challenging environments, such as ultradeep water and hard rock formations. The machine learning algorithms, applied to real-time analysis, has occupied an important role in the oil well industry. We here propose a machine learning system to aid drill bit wear assessment for real-time operations, using a Probabilistic Neural Network (PNN) and Bayes Theorem, combined with Power Spectral Density (PSD) of surface torque image and feature extraction by Deep Convolutional Autoencoder (DCAE). A case study was performed with wells from Brazilian ultradeep water pre-salt fields. The analysis shows that a feature extraction DCAE is more efficient to discriminate the bit wear state when compared with PSD torque image and torque raw data. After that, several numerical experiments were performed to investigate the best machine learning classification model, considering DCAE feature extraction. The cross-validation results considering all the dataset shows the PNN evaluation metrics: 87% accuracy, 83% precision, 91% recall and 86% for F1-score. The field tests validation was performed in real-time simulation, with 3 wells from the original dataset plus 2 offset wells. The simulations show that the proposed system can capture the best moment for the tripping pipe to replace a worn-out drill bit. The field tests experimental results indicate that the system proposed can aid drilling engineers in monitoring Polycrystalline Diamond Compact (PDC) drill bit wear in real-time operations.

Real-time filtering on parallel SIMD architectures for automated quality inspection

Surface metrology in automated quality inspection is a field, among many others, affected by noise and thus requiring filtering. In surface metrology, filtering is required to remove undesired information from data in order to extract surface features and relevant properties necessary for quality control. Moreover, filtering requires immediate results, while the product is being manufactured. This way, quick correcting actions can be directly applied to solve possible manufacturing issues. This work proposes different strategies to filter height maps in real-time acquired using laser profilers, the most widely used inspection method in industrial applications. Different models to apply the filtering operations are considered, particularly assessing different alternatives to store previous samples in memory, which are required for data filtering. FIFO, double FIFO, circular and double circular buffers are evaluated. Furthermore, CPU parallelism, SIMD instructions and cache-line friendly data structures are analyzed. The proposed methods are extremely efficient, capable of filtering laser profiles at extremely high acquisition rates. The proposed methods are designed for real-time surface metrology, but they are very likely to find potential applications in different areas. The filters are compared in terms of accuracy and speed, including other well-known filters such as the spline filter. Tests analyze execution time, including cache efficiency and filtering accuracy. Results with synthetic data and real data obtained from steel strips show excellent performance, providing accurate results at very high speeds.

Depth Image-Based Deep Learning of Grasp Planning for Textureless Planar-Faced Objects in Vision-Guided Robotic Bin-Picking.

Bin-picking of small parcels and other textureless planar-faced objects is a common task at warehouses. A general color image–based vision-guided robot picking system requires feature extraction and goal image preparation of various objects. However, feature extraction for goal image matching is difficult for textureless objects. Further, prior preparation of huge numbers of goal images is impractical at a warehouse. In this paper, we propose a novel depth image–based vision-guided robot bin-picking system for textureless planar-faced objects. Our method uses a deep convolutional neural network (DCNN) model that is trained on 15,000 annotated depth images synthetically generated in a physics simulator to directly predict grasp points without object segmentation. Unlike previous studies that predicted grasp points for a robot suction hand with only one vacuum cup, our DCNN also predicts optimal grasp patterns for a hand with two vacuum cups (left cup on, right cup on, or both cups on). Further, we propose a surface feature descriptor to extract surface features (center position and normal) and refine the predicted grasp point position, removing the need for texture features for vision-guided robot control and sim-to-real modification for DCNN model training. Experimental results demonstrate the efficiency of our system, namely that a robot with 7 degrees of freedom can pick randomly posed textureless boxes in a cluttered environment with a 97.5% success rate at speeds exceeding 1000 pieces per hour.

Adaptive control of the filler wire rate during laser beam welding of squared butt joints with varying gap width

Adding filler wire control to autogenous laser beam welding of squared butt joints offers a means to widen up the tight fit-up tolerances associated with this process. When the gap width varies, the filler wire rate should be controlled to assure a constant geometry of the resulting weld seam. A dual mode sensing system is proposed to estimate the joint gap width and thereby control the filler wire rate. A vision camera integrated into the welding tool together with external LED illumination and a laser line projection enables two sensing modes, one surface feature extraction mode and one laser triangulation-based mode. Data from the both modes are fused in a Kalman filter, and comparisons show that the fusing of the data gives more robust estimation than estimates from each single mode. A feed-forward control system adaptively adjusts the filler wire rate based on the estimations of the joint gap width in front of the keyhole. The focus is on keeping the data processing simple and affordable, and the real-time performance of the sensor and control system has been evaluated by welding experiments. It is shown that the proposed system can be used for on-line control of the filler wire rate to achieve a constant weld geometry during varying joint gap widths.

Three-dimensional reconstruction of wear particle surface based on photometric stereo

Ferrography has emerged as a significant candidate for the provision of relevant information for the determination of the status of machine wear. However, this conventional methodology provides only marginal results due to its inability to provide 3-dimensinal (3D) surface information. The authors of this paper have developed a methodology based on photometric-stereo towards the enhancement of the capabilities of ferrography. This enhanced and innovative methodology consists of three main components, the multi-illumination image acquisition, the wear particle extraction, and the 3D surface reconstruction. The methodology ensures the reliable and efficient extraction of the wear particles surface topographies for further feature-based wear particle identification. The performance of this methodology has been compared with results observed from the laser scanning confocal microscopy. The outcome of this comparison has depicted that the methodology involving this new low-cost ferrography system exhibits very high accuracy for the 3D surface feature extraction of wear particles.

Surface electromyography segmentation and feature extraction for ingestive behavior recognition in ruminants

This work presents a method to identify the ingestive behavior in ruminants using Surface Electromyography (sEMG) of the masseter muscle. The main hypothesis tested is whether the rumination and food eaten can be recognized from sEMG signal features using machine learning techniques. Also, a novel segmentation technique was explored and applied to automatically subdivide the chewing movement signal. Seven classifiers were evaluated using eight features extracted from the signal and combined into five sets. The three scenarios investigated were: differentiation between rumination and grazing (IR), food identification for four different foods (FC) and both situations combined (FCR). The segmentation window size effect on the accuracy was also investigated. We found an accuracy over 70% for IR and FC and nearby 60% for FCR using a Multilayer Perceptron Neural Network (MLP-NN). Highlighted features were the Cepstrum coefficients (CEPS) and the signal Wavelength (WL). Segments between 600 and 1000 ms proved to be suitable. The segmentation technique and the proposed scheme are reliable for application although there is room for improvement.

The Utility of Sentinel-1 Data for Ocean Surface Feature Analysis in the Vicinity of the Gulf Stream

ABSTRACTEuropean Space Agency Sentinel-1 (S1) synthetic aperture radar (SAR) datasets are used to assess their suitability for ocean surface feature extraction in the vicinity of the Gulf Stream. A SAR ocean feature detection tool originally developed to extract and classify brightness fronts in RADARSAT-2 (R2) SAR imagery has been updated to extract and classify brightness fronts from S1 ground range-detected products. Results indicate that the features extracted from S1 datasets are largely consistent with those derived from R2. However, more features are extracted from S1 data and, in limited examples, there are differences in how features are classified. In addition, S1 radial velocity (RVL) products were compared with simulated RVL products created from modeled ocean current data. This comparison shows that the orientation and magnitude of the Gulf Stream in S1 RVL products is generally consistent with modeled results, but does indicate that some regions of the Gulf Stream (meanders and ocean eddies) are likely to present challenges for automatic extraction algorithms. Taken together, this research shows that S1 data have utility for ocean surface feature extraction and provide an additional dataset that can be exploited to expand ocean feature analysis over Canadian waters.