Density-based Clustering Research Articles

Data-driven approach to optimizing property management strategies: spatial modeling analytics of short-term rentals

Purpose This study aims to explore the impact of locational and seasonal factors on the financial performance of short-term rental properties in Margaret River, Western Australia. It seeks to address the gap in understanding how these factors influence key financial metrics such as average daily rate (ADR) and occupancy rates, providing insights for property managers, investors and policymakers. Design/methodology/approach The research uses a mixed-method approach, integrating advanced predictive modeling techniques, such as Random Forests and Gradient Boosting, with spatial clustering algorithms like density-based spatial clustering of applications with noise (DBSCAN) and ordering points to identify the clustering structure (OPTICS). The study analyzes a comprehensive data set of short-term rental properties between 2012 and 2019. It focuses on locational attributes, seasonal variations and financial outcomes. Findings The findings reveal that properties located near tourist attractions and amenities consistently achieve higher ADRs and occupancy rates, confirming the critical role of location in driving rental demand. Seasonal analysis indicates significant fluctuations in both ADR and occupancy rates, with peaks during high tourist seasons and troughs in off-peak periods. The study underscores the importance of dynamic pricing strategies to optimize revenue and sustain occupancy across different seasons. In addition, it highlights the influence of property features, such as the number of bedrooms and bathrooms, on ADR, while noting that larger properties do not necessarily achieve higher occupancy rates. Research limitations/implications Future research could expand the scope to include different locations and explore the long-term impacts of locational and seasonal factors on property performance. Originality/value This research contributes to the literature by integrating spatial analysis with advanced predictive modeling techniques to provide a nuanced understanding of how locational and seasonal factors impact financial performance in the short-term rental market. It offers a novel application of data analytics within the context of tourism and hospitality management, bridging theoretical frameworks with practical insights.

Robust Parameter Optimisation of Noise-Tolerant Clustering for DENCLUE Using Differential Evolution

Clustering samples based on similarity remains a significant challenge, especially when the goal is to accurately capture the underlying data clusters of complex arbitrary shapes. Existing density-based clustering techniques are known to be best suited for capturing arbitrarily shaped clusters. However, a key limitation of these methods is the difficulty in automatically finding the optimal set of parameters adapted to dataset characteristics, which becomes even more challenging when the data contain inherent noise. In our recent work, we proposed a Differential Evolution-based DENsity CLUstEring (DE-DENCLUE) to optimise DENCLUE parameters. This study evaluates DE-DENCLUE for its robustness in finding accurate clusters in the presence of noise in the data. DE-DENCLUE performance is compared against three other density-based clustering algorithms—DPC based on weighted local density sequence and nearest neighbour assignment (DPCSA), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), and Variable Kernel Density Estimation–based DENCLUE (VDENCLUE)—across several datasets (i.e., synthetic and real). The study has consistently shown superior results for DE-DENCLUE compared to other models for most datasets with different noise levels. Clustering quality metrics such as the Silhouette Index (SI), Davies–Bouldin Index (DBI), Adjusted Rand Index (ARI), and Adjusted Mutual Information (AMI) consistently show superior SI, ARI, and AMI values across most datasets at different noise levels. However, in some cases regarding DBI, the DPCSA performed better. In conclusion, the proposed method offers a reliable and noise-resilient clustering solution for complex datasets.

A Novel Non-Contact Multi-User Online Indoor Positioning Strategy Based on Channel State Information

The IEEE 802.11bf-based wireless fidelity (WiFi) indoor positioning system has gained significant attention recently. It is important to recognize that multi-user online positioning occurs in real wireless environments. This paper proposes an indoor positioning sensing strategy that includes an optimized preprocessing process and a new machine learning (ML) method called NKCK. The NKCK method can be broken down into three components: neighborhood component analysis (NCA) for dimensionality reduction, K-means clustering, and K-nearest neighbor (KNN) classification with cross-validation (CV). The KNN algorithm is particularly suitable for our dataset since it effectively classifies data based on proximity, relying on the spatial relationships between points. Experimental results indicate that the NKCK method outperforms traditional methods, achieving reductions in error rates of 82.4% compared to naive Bayes (NB), 85.0% compared to random forest (RF), 72.1% compared to support vector machine (SVM), 64.7% compared to multilayer perceptron (MLP), 50.0% compared to density-based spatial clustering of applications with noise (DBSCAN)-based methods, 42.0% compared to linear discriminant analysis (LDA)-based channel state information (CSI) amplitude fingerprinting, and 33.0% compared to principal component analysis (PCA)-based approaches. Due to the sensitivity of CSI, our multi-user online positioning system faces challenges in detecting dynamic human activities, such as human tracking, which requires further investigation in the future.

Machine Learning Clustering Techniques to Support Structural Monitoring of the Valgadena Bridge Viaduct (Italy)

The lack of precise and comprehensive information about the health of bridges, and in particular long span ones, can lead to incorrect decisions regarding maintenance, repair, modernization, and reinforcement of the structure itself. While the consequences of inadequate interventions are quite apparent, incorrect decisions can also result in unnecessary or misdirected actions. For example, an inadequate assessment of the structural health can lead to the modernization and replacement of some components that are still sound. Structural Health Monitoring (SHM) involves the use of time series derived from periodic measurements of the structure’s behavior, considered in its operational and load environment. The goal is to determine its response to various solicitations and, in particular, to highlight any critical issue in the structure’s behavior that may affect its reliability and safety due to anomalies and deterioration. This paper proposes an SHM method applied to the Valgadena bridge, one of the tallest viaducts in Italy and Europe (maximum height 160 m), located on the Altopiano dei Sette Comuni in the Province of Vicenza. Despite the fact that the viaduct itself had already been monitored during its construction using classical geometric leveling techniques, the methodology proposed here is based instead on the use of affordable dual-frequency GNSS (Global Navigation Satellite System) receivers to determine static and dynamic components of the bridge movements. Specifically, an effective combination of time series analysis methods and machine learning techniques is proposed in order to determine the vibration modes of the monitored viaduct. Monitoring is performed in regular operation conditions of the bridge (operational modal analysis (OMA)), and the use of certain machine learning methods aims at supporting the development of an effective automatic OMA procedure. To be more specific, the random decrements technique is used in order to make the vibration characteristics of the collected signals more apparent. Time-domain-based subspace identification is applied in order to determine a proper model of the collected measurements. Then, clustering methods, namely DBSCAN (Density-Based Spatial Clustering of Applications with Noise) and GMMs (Gaussian Mixture Models), are used in order to reliably estimate the system poles, and hence the corresponding vibration characteristics. The performance of the considered methods is compared on the Valgadena bridge case study, showing that the use of GMM clustering reduces, with respect to DBSCAN, the impact of the choice of certain parameter values in the considered case.

Machine learning approach to flare-up detection and clustering in chronic obstructive pulmonary disease (COPD) patients.

Chronic obstructive pulmonary disease (COPD) is a prevalent and preventable condition that typically worsens over time. Acute exacerbations of COPD significantly impact disease progression, underscoring the importance of prevention efforts. This observational study aimed to achieve two main objectives: (1) identify patients at risk of exacerbations using an ensemble of clustering algorithms, and (2) classify patients into distinct clusters based on disease severity. Data from portable medical devices were analyzed post-hoc using hyperparameter optimization with Self-Organizing Maps (SOM), Density-Based Spatial Clustering of Applications with Noise (DBSCAN), Isolation Forest, and Support Vector Machine (SVM) algorithms, to detect flare-ups. Principal Component Analysis (PCA) followed by KMeans clustering was applied to categorize patients by severity. 25 patients were included within the study population, data from 17 patients had the required reliability. Five patients were identified in the highest deterioration group, with one clinically confirmed exacerbation accurately detected by our ensemble algorithm. Then, PCA and KMeans clustering grouped patients into three clusters based on severity: Cluster 0 started with the least severe characteristics but experienced decline, Cluster 1 consistently showed the most severe characteristics, and Cluster 2 showed slight improvement. Our approach effectively identified patients at risk of exacerbations and classified them by disease severity. Although promising, the approach would need to be verified on a larger sample with a larger number of recorded clinically verified exacerbations.

Point-cloud clustering and tracking algorithm for radar interferometry

In data mining, density-based clustering, which entails classifying datapoints according to their distributions in some space, is an essential method to extract information from large datasets. With the advent of software-based radio, ionospheric radars are capable of producing unprecedentedly large datasets of plasma turbulence backscatter observations, and new automatic techniques are needed to sift through them. We present an algorithm to automatically identify and track clusters of radar echoes through time, using , a celebrated density-based clustering method for noisy point clouds. We demonstrate our algorithm's efficiency by tracking turbulent structures in the E-region ionosphere, the so-called radar aurora. Through conjugate auroral imagery, as well as satellite observations, we demonstrate that the observed turbulent structures generally track the motion of auroras. What is more, the radar aurora bulk motions exhibit key qualities of auroral electric field enhancements that have previously been observed with various instruments. We present preliminary statistical results using our method, and briefly discuss the method's limitations and potential future adaptations. Published by the American Physical Society 2024

Unsupervised Clustering-based 3D Static Scene Construction Using LiDAR Channel and Azimuth Angle

Abstract. Cameras are typically used at road intersections to collect data to perform object detection but struggle in low-light and harsh weather. On the other hand, Light Detection and Ranging (LiDAR) is used as a key technology in 3D vision systems. It gives the 3D point cloud, which includes accurate depth information, but its resolution is cost-dependent, with higher resolutions being more expensive. Deep learning-based method requires large, labelled dataset which increases the cost, time and accuracy depending on the model trained on labelled dataset. To perform object detection in point cloud data is a challenging task due to the incomplete representations, data sparsity and unavailability of training data. To overcome this by using an unsupervised approach, it is important to identify the static scene and then detect the moving object. This work incorporated a novel approach to construct static scene using azimuth angle and laser channel information using an unsupervised clustering approach. This work incorporates two modules I) data collection using VLP-16 LiDAR, and II) static scene construction using DBSCAN (density-based spatial clustering and noise) clustering-based approach. Data is collected at a 4-legged intersection and pre-processed to extract aggregated distances corresponding to the unique pair of azimuth angle and laser channel. DBSCAN is used to perform clustering on the aggregated distances, based on the highest silhouette score and lowest intra distance between points in cluster, static points are identified, and static scene constructed. The qualitative evaluation of method demonstrates that the algorithm effectively and accurately filters out background points.

Discovering Electric Vehicle Charging Locations Based on Clustering Techniques Applied to Vehicular Mobility Datasets

With the proliferation of vehicular mobility traces because of inexpensive on-board sensors and smartphones, utilizing them to further understand road movements have become easily accessible. These huge numbers of vehicular traces can be utilized to determine where to enhance road infrastructures such as the deployment of electric vehicle (EV) charging stations. As more EVs are plying today’s roads, the driving anxiety is minimized with the presence of sufficient charging stations. By correctly extracting the various transportation parameters from a given dataset, one can design an adequate and adaptive EV charging network that can provide comfort and convenience for the movement of people and goods from one point to another. In this study, we determined the possible EV charging station locations based on an urban city’s vehicular capacity distribution obtained from taxi and ride-hailing mobility GPS traces. To achieve this, we first transformed the dynamic vehicular environment based on vehicular capacity into its equivalent urban single snapshot. We then obtained the various traffic zone distributions by initially utilizing k-means clustering to allow flexibility in the total number of wanted traffic zones in each dataset. In each traffic zone, iterative clustering techniques employing Density-based Spatial Clustering of Applications with Noise (DBSCAN) or clustering by fast search and find of density peaks (CFS) revealed various area separation where EV chargers were needed. Finally, to find the exact location of the EV charging station, we last ran k-means to locate centroids, depending on the constraint on how many EV chargers were needed. Extensive simulations revealed the strengths and weaknesses of the clustering methods when applied to our datasets. We utilized the silhouette and Calinski–Harabasz indices to measure the validity of cluster formations. We also measured the inter-station distances to understand the closeness of the locations of EV chargers. Our study shows how CFS + k-means clustering techniques are able to pinpoint EV charger locations. However, when utilizing DBSCAN initially, the results did not present any notable outcome.

Adaptive sensor attack detection and defense framework for autonomous vehicles based on density

The security of autonomous vehicles heavily depends on localization systems that integrate multiple sensors, which are vulnerable to sensor attacks and increase the risk of accidents. Given the diversity of sensor attacks and the dynamic changing of driving scenarios of autonomous vehicles, an adaptive and effective attack detection and defense framework faces a considerable challenge. This paper proposes a novel real-time adaptive attack detection and defense framework based on density, which can detect and identify attacked sensors and effectively recover data. We first develop a reinforcement learning multi-armed Bandit-based Density-Based Spatial Clustering of Applications with Noise (BDBSCAN) algorithm that selects hyperparameters adaptively. The Adaptive Extended Kalman Filter (AEKF) combines with the vehicle dynamic model on the localization system and extracts data features used for the BDBSCAN algorithm to monitor potential sensor attacks. If attack detection indicates possible system compromise, AEKF is further employed on localization sensors with anomalies identified through the BDBSCAN algorithm of the attacked sensors. To ensure precision and reliability, the data recovery incorporates a redundancy mechanism to apply a decision tree to select the optimal state estimation between AEKF and Extended Kalman Filter (EKF) to replace corrupted sensor data. To evaluate the effectiveness and adaptability of the proposed framework, we conducted 15,000 experiments using the real-world KITTI and V2V4Real datasets across various driving and sensor attack scenarios. The results demonstrate that our proposed framework achieves 100% accuracy and 0% false alarm rate in various driving scenarios for attack detection within 0.15 s, with a recovery time of 0.08 s.

Uncertain data density peak clustering algorithm based on JS divergence

Aiming at the defects of traditional density-based uncertainty clustering algorithms, such as parameter sensitivity and poor clustering results for complex manifold uncertain data sets, a new uncertainty data density peak clustering algorithm based on JS divergence (UDPC-JS) is proposed. The algorithm first removes noise points by using the uncertain natural neighbor density factor defined by the uncertain natural neighbor. Secondly, the local density of uncertain data objects is calculated by combining the uncertain natural neighbor and JS divergence. The initial clustering center of the uncertain data set is found by combining the idea of representative points, and the distance between the initial clustering centers is defined based on JS divergence and graph. Then, the decision graph is constructed on the initial clustering center using the local density calculated based on the uncertain natural neighbor and JS divergence and the newly defined distance between the initial clustering centers based on JS divergence and graph, and the final clustering center is selected according to the decision graph. Finally, the unassigned uncertain data objects are assigned to the cluster where their initial clustering center is located. Experimental results show that the algorithm has better clustering effect and accuracy than the comparison algorithm, and has a greater advantage in processing complex manifold uncertain data sets.

DPC clustering algorithm based on K-nearest neighbors and kernel density estimation

Clustering by fast search and find of density peaks (DPC) is a density-based clustering algorithm. This algorithm does not require iteration and too many parameter settings, but it cannot identify cluster centers with small cluster density because the local structure of the data is not considered when calculating the local density. To address the above problems, a DPC clustering algorithm (KKDPC) based on K-reciprocal Neighbors (KN) and Kernel Density Estimation (KDE) is proposed. First, the number of reciprocal neighbors and local kernel density of data points are obtained by nearest neighbor and kernel density estimation methods ; second, the number of K reciprocal neighbors is added to the local kernel density to obtain a new local density; finally, the relative distance is obtained according to the local density of the data points, and the cluster center is selected and the non-center point is allocated by constructing a decision graph. Experiments are conducted using artificial and real data sets, and compared with seven clustering algorithms: DPC, DBSCAN, K-means, FCM, DPC-KNN, DPC-NN, and DPC-FWSN. The performance of the KKDPC clustering algorithm is verified by calculating the adjusted mutual information (AMI), adjusted Rand index (ARI), and normalized mutual information (NMI). The experimental results show that this method can more accurately identify the cluster center and effectively improve the clustering accuracy.

Interpretable multi-omics integration with UMAP embeddings and density-based clustering.

Integrating high-dimensional cellular multi-omics data is crucial for understanding various layers of biological control. Single 'omic methods provide important insights, but often fall short in handling the complex relationships between genes, proteins, metabolites and beyond. Here, we present a novel, non-linear, and unsupervised method called GAUDI (Group Aggregation via UMAP Data Integration) that leverages independent UMAP embeddings for the concurrent analysis of multiple data types. GAUDI uncovers non-linear relationships among different omics data better than several state-of-the-art methods. This approach not only clusters samples by their multi-omic profiles but also identifies latent factors across each omics dataset, thereby enabling interpretation of the underlying features contributing to each cluster. Consequently, GAUDI facilitates more intuitive, interpretable visualizations to identify novel insights and potential biomarkers from a wide range of experimental designs.

High-Speed Fluctuation Analysis of Silver-Nanoparticle SERS in Solutions.

We analyzed the fluctuation of surface-enhanced Raman spectra with a temporal resolution of 25 ms using a conventional electron-multiplying charge-coupled device camera experimental setup. The signal-to-noise ratio of the spectra was improved using density-based spatial cluster analysis with noise. Silver nanoparticles (AgNPs) with different sizes were dispersed as surface-enhanced Raman spectroscopy platforms in violet aqueous solutions. The movement of AgNPs and the fluctuation of the spectra were characterized. The fluctuation (signal ON and OFF) was evaluated on the basis of the time intervals between ON and OFF timing. The behavior of each AgNP solution was explained by a two-dimensional random walk model, which means that the phenomenon was mainly governed by the Brownian motion of the AgNPs in the solution. The fluctuation was also compared among three different Raman modes, one of which showed anomalous behavior.

Unsupervised Learning for Lateral-Movement-Based Threat Mitigation in Active Directory Attack Graphs

Cybersecurity threats, particularly those involving lateral movement within networks, pose significant risks to critical infrastructures such as Microsoft Active Directory. This study addresses the need for effective defense mechanisms that minimize network disruption while preventing attackers from reaching key assets. Modeling Active Directory networks as a graph in which the nodes represent the network components and the edges represent the logical interactions between them, we use centrality metrics to derive the impact of hardening nodes in terms of constraining the progression of attacks. We propose using Unsupervised Learning techniques, specifically density-based clustering algorithms, to identify those nodes given the information provided by their metrics. Our approach includes simulating attack paths using a snowball model, enabling us to analytically evaluate the impact of hardening on delaying Domain Administration compromise. We tested our methodology on both real and synthetic Active Directory graphs, demonstrating that it can significantly slow down the propagation of threats from reaching the Domain Administration across the studied scenarios. Additionally, we explore the potential of these techniques to enable flexible selection of the number of nodes to secure. Our findings suggest that the proposed methods significantly enhance the resilience of Active Directory environments against targeted cyber-attacks.

A Time-Frequency Domain Analysis Method for Variable Frequency Hopping Signal.

A variable frequency hopping (VFH) signal is a kind of frequency hopping (FH) signal that varies both in frequency and dwell time. However, in radio surveillance, the existing methods for unidentified signals using VFH cannot be effectively handled. In this paper, we proposed an improved joint analysis method based on time-frequency domain features, which adopts multi-level processing to solve the time-frequency domain feature analysis problem of the VFH signal. First, the received signal is pre-processed by Short-Time Fourier Transform (STFT) and binarization, and a highly discriminative time-frequency image is obtained; then, the fixed frequency signal is removed based on the feature of connected domains, and the conventional frequency hopping (CFH) signal is removed by density-based spatial clustering of applications with noise (DBSCAN); finally, the overlapping region is cropped by the joint energy peak time-domain continuity properties. After the above multi-level joint processing method, the problem of VFH signal processing is effectively solved. The simulation result shows that the Mean Square Error (MSE) between the output results and the time-frequency image of the original VFH signal tends to be close to 0 when the Signal-to-Noise ratio (SNR) is 5 dB.

Fault diagnosis of planetary gearboxes under small and imbalanced samples based on enhanced Siamese network with improved downsampling module

Due to the limitations of the operating conditions and equipment status, in practice, the normal data for planetary gearboxes are often sufficient, but the fault data are scarce and difficult to obtain, causing the fault diagnosis to face not only an imbalance problem but also a small sample problem. It leads to serious misdiagnosis and poor generalizability. To overcome above challenges, an enhanced Siamese network with improved downsampling module is proposed to reduce the negative impact of small and imbalanced datasets on fault diagnosis. Firstly, the Siamese network is combined with data enhancement to adaptively increase the fault training data with different small sample degrees. It can make the number of training data of each class get rid of small sample, while avoiding the excessive use of scarce fault data causing additional overfitting, so as to improve the generalization of the model. Among them, the data enhancement adopts the extreme noise expansion method proposed in this work, which can quickly generate samples that meet the requirements and help to improve the performance of the model in noisy environments. Secondly, adding the improved downsampling module to the Siamese network. The improved downsampling module introduces density-based spatial clustering of applications with noise (DBSCAN) and distribution ratio calculation, which can consider the distribution range and density of data when selecting. It can not only balance the training data, but also avoid serious information loss, and then reduce the probability of misdiagnosis. Finally, using six imbalanced planetary gearbox datasets with different small sample degrees for verification. The experimental results show that the proposed method can efficiently increase and balance the training data, which greatly improves the accuracy of fault diagnosis under the condition of small and imbalanced samples, especially when the small sample degree is very serious, the accuracy is improved by 57.76% on average.

Nanoscale organization of cardiac calcium channels is dependent on thyroid hormone status.

Thyroid hormone dysfunction is frequently observed in patients with chronic illnesses including heart failure which increases risk of adverse events. This study examined effects of thyroid hormones (TH) on cardiac T-tubule (TT) integrity, Ca2+ sparks, and nanoscale organization of ion channels in excitation-contraction (EC)-coupling, including L-type calcium channel (Cav1.2), ryanodine receptor-type 2 (RyR2), and junctophilin-2 (Jph2). TH deficiency was established in adult female rats by propyl-thiouracil (PTU) ingestion for 8 weeks; followed by randomization to continued PTU without or with oral triiodo-L-thyronine (T3; 10 ug/kg/d) for two additional weeks (PTU+T3). Confocal microscopy of isolated cardiomyocytes (CM) showed significant misalignment of TTs, and increased Ca2+ sparks in thyroid-deficient CMs. Density-Based Spatial Clustering of Applications with Noise (DBSCAN) analysis of STochastic Optical Reconstruction Microscopy (STORM) images showed decreased (p<0.0001) RyR2 cluster number per cell area in PTU CMs compared to euthyroid (EU) control myocytes, and this was normalized by T3-treatment. Cav1.2 channels and Jph2 localized within 210 nm radius of the RyR2 clusters were significantly reduced in PTU myocytes, and these values were increased with T3 treatment. A significant percentage of the RyR2 clusters in the PTU myocytes had neither Cav1.2 or Jph2, suggesting fewer functional clusters in EC-coupling. Nearest neighbor distances between RyR2 clusters were greater (p<0.001) in PTU cells compared to EU and T3-treated CMs that corresponds to disarray of TTs at the sarcomere z-discs. These results support a regulatory role of T3 in the nanoscale organization of RyR2 clusters and co-localization of Cav1.2 and Jph2 in optimizing EC-coupling.

An aero-engine remaining useful life prediction model based on clustering analysis and the improved GRU-TCN

Abstract Accurately predicting the remaining useful life (RUL) of engines is paramount for implementing effective preventive maintenance strategies, preventing injuries and fatalities caused by equipment failures, and significantly reducing routine repair and replacement costs. However, existing deep learning models often ignore the variable operating conditions in real engineering applications and do not sufficiently consider the interaction between time series and degradation laws, which directly leads to the inability to effectively extract to degradation feature extraction. To address this problem, this study developed a novel combined network model named CA-DRGRU-TTCN, aimed at accurately predicting the RUL of engines. Firstly, a Density-Based Spatial Clustering of Applications with Noise (DBSCAN) algorithm is used to identify multiple operating conditions, and incorporate the recognition results into the model as additional new features. The first degradation time point is determined by JS divergence. Secondly, the deep connectivity of the residual Gated Recurrent Unit (GRU) module is designed to extract deeper degradation features, and an improved TMSE loss function based on the first degradation time point is applied to Temporal Convolutional Networks (TCN) to better capture the dependency between the time series and the real degradation degree of the engine. Finally, experiment results on the C-MAPSS dataset show that the proposed method achieves better performance compared to existing state-of-the-art methods.&amp;#xD;

NS-IDBSCAN: An efficient incremental clustering method for geospatial data in network space

The exponential growth of big data presents a significant task for the incremental clustering problem. This paper proposes a density-based total clustering method in network space (NS-IDBSCAN) to cluster newly generated data. Instead of re-clustering the entire massive database, which would be much larger than the new datasets, we perform clustering with only the added data. Based on the current clustering results of the old data, the proposed algorithm checks the role of each newly added point and its neighbors in performing clustering. Depending on the density, an added point can be classified as noise, border, or core. This approach dramatically reduces the response time. Moreover, using a hash table with the key as the element's index corresponding to the point Id to store the cluster Id eliminates the need for searching by direct access, further increasing processing speed. Additionally, the proposed algorithm improves accuracy by reducing the intra-cluster distance when eliminating the phenomenon that the border points may belong to more than one cluster, as presented in the discussion. The proposed algorithm is compared with the spatial data clustering algorithm in network space. Test results on three data sources downloaded from OpenStreetMap, ESRI Open Data, and Inside Airbnb demonstrate that the proposed method significantly speeds up the processing time. The accuracy of the proposed algorithm is measured using eight indicators: Silhouette, BSS, WSS, WB, Davis-Bouldin, Dunn, Calinski-Harabasz, and NMI.

Automated matching and visualisation of magnetic flux leakage data in shale gas pipeline based on ICP and DBSCAN algorithm

ABSTRACT Magnetic flux leakage (MFL) is a non-destructive testing method for shale gas pipeline safety monitoring. Despite many pipelines have completed two or more rounds of internal inspection and have accumulated substantial data, effectively analyzing and utilizing this MFL data remains a significant challenge. To that end, this study proposes a novel combined model that integrates Iterative Closest Point (ICP) and DBSCAN (Density-Based Spatial Clustering of Applications with Noise) fusion to match multiple MFL data. The model outputs the matching result and visualisation images of MFL data acquired in different years. Furthermore, RMSE (root mean square error) and overlap rate have been used to evaluate the model. The results indicate that the average distance error of matched defects between two datasets decreased by 72.5%, and the overlap rate increased by 20%. Additionally, the DBSCAN Fusion shows better computational efficiency with an increase in defect quantity within the pipe segments. Finally, the impact of different datasets on the model’s matching accuracy is discussed in this study. The findings show that small-scale datasets or higher false detection rates lead to decreased accuracy. The proposed model fully leverages MFL data to achieve rapid matching of defects, offering a dependable and effective technical solution for safety monitoring and corrosion prediction in shale gas pipelines.