Density-based Spatial Clustering Algorithm Research Articles

Klasifikasi Penggunaan Teknologi Pada Petani Milenial di Sulawesi Selatan Menggunakan Density Based Spatial Clustering Algorithm With Noise

This research examines the use of agricultural technology by millennial farmers in South Sulawesi Province by applying the Density-Based Spatial Clustering Algorithm with Noise (DBSCAN) to the agricultural census data of South Sulawesi Province Phase 1 in 2023. The secondary data used includes the number of millennial farmers aged 19-39 who use or do not use digital technology, divided by district/city and gender. The analysis process begins with preprocessing to prepare the data, followed by clustering using the DBSCAN algorithm, determining the optimal values for the Eps and minPts parameters, and evaluating the quality of the formed clusters using the silhouette coefficient and elbow method. The results of the study indicate that the combination of Eps value of 1.000 and minPts value of 7 produces optimal clustering with 2 clusters formed and 92 data points clustered, while 4 other data points are considered as noise. Evaluation using the silhouette coefficient and elbow method also indicates that the optimal data grouping is k=2.

Comparison of two novel methods for counting wheat ears in the field with terrestrial LiDAR

BackgroundThe metrics for assessing the yield of crops in the field include the number of ears per unit area, the grain number per ear, and the thousand-grain weight. Typically, the ear number per unit area contributes the most to the yield. However, calculation of the ear number tends to rely on traditional manual counting, which is inefficient, labour intensive, inaccurate, and lacking in objectivity. In this study, two novel extraction algorithms for the estimation of the wheat ear number were developed based on the use of terrestrial laser scanning (TLS) in conjunction with the density-based spatial clustering (DBSC) algorithm based on the normal and the voxel-based regional growth (VBRG) algorithm. The DBSC involves two steps: (1) segmentation of the point clouds using differences in the normal vectors and (2) clustering of the segmented point clouds using a density clustering algorithm to calculate the ear number. The VBRG involves three steps: (1) voxelization of the point clouds, (2) construction of the topological relationships between the voxels as a connected region using the k-dimensional tree, and (3) detection of the wheat ears in the connected areas using a regional growth algorithm.ResultsThe results demonstrated that DBSC and VBRG were promising in estimating the number of ears for different cultivars, planting densities, N fertilization rates, and growth stages of wheat (RMSE = 76 ~ 114 ears/m2, rRMSE = 18.62 ~ 27.96%, r = 0.76 ~ 0.84). Comparing the performance of the two algorithms, the overall accuracy of the DBSC (RMSE = 76 ears/m2, rRMSE = 18.62%, r = 0.84) was better than that of the VBRG (RMSE = 114 ears/m2, rRMSE = 27.96%, r = 0.76). It was found that with the DBSC, the calculation in points as units permitted more detailed information to be retained, and this method was more suitable for estimation of the wheat ear number in the field.ConclusionsThe algorithms adopted in this study provide new approaches for non-destructive measurement and efficient acquisition of the ear number in the assessment of the wheat yield phenotype.

Efficient fracture network characterization in enhanced geothermal reservoirs by the integration of microseismic and borehole logs data

Characterization of 3D fracture networks induced by hydraulic fracturing in enhanced geothermal reservoirs is crucial for safe and effective energy recovery from deep geothermal reservoirs. Microseismic monitoring is one of the most promising ways to infer the geometry of fracture network. However, the identification of fracture network from microseismic events is still a challenging task, attributed to high noises in microseismicity interpretation and fuzzy relationships between microseismic event locations and fracture planes. This study proposed a novel methodology to identify fracture network from the spatial distribution of microseismic events. The method was composed by (1) density-based spatial clustering algorithm for microseismic data denoising, (2) Monte Carlo optimization algorithm for fractures localization by minimizing the total distance between denoised microseismic events and fracture planes, and (3) Elbow method to decide the most likely number of fractures. Moreover, the method employed the prior knowledge of geo-stresses and initial fracture patterns inferred from borehole logs to constrain the geometry of fracture network. Following verifications with synthetic fractures and microseismic events, the method was implemented in a realistic enhanced geothermal system, which reliably identified the geometry of 3D fracture network induced by hydraulic fracturing.

A Robust Optimizing Reverse Logistics Model for Beef Products Using Multi Depot Vehicle Routing Problem

Beef is a perishable product and requires special handling. Demand for beef also fluctuates quite high and is heavily influenced by various religious events and traditions in Indonesia. Under these conditions, for various reasons, beef products are returned from customers to distributors. An increase in the number of products returned from customers leads to high costs and the risk of product damage. The research created an optimization model for product distribution and product recall from customers with minimal costs and risks. The research applied a clustering method using the Density-Based Spatial Clustering (DBSCAN) algorithm to determine the density of customers’ locations and the number of orders. Optimization of distance and distribution and withdrawal costs applied Multi Depot Vehicle Routing Problem (MDVRP) and Mixed Integer Linear Programming (MILP) mathematical modeling. The results indicate three customer clusters with one noise, with the most potential customers in cluster 1. From this condition, product delivery optimization is based on the distance and number of shipments from the two central warehouses. Optimization uses of MDVRP and MILP to model and make company-owned trucks more profitable at high rental truck replacement costs. The research produces a robust model for changes in the truck number and capacity based on sensitivity analysis.

Sparse Time-Frequency Distribution Reconstruction Using the Adaptive Compressed Sensed Area Optimized with the Multi-Objective Approach.

Compressive sensing (CS) of the signal ambiguity function (AF) and enforcing the sparsity constraint on the resulting signal time-frequency distribution (TFD) has been shown to be an efficient method for time-frequency signal processing. This paper proposes a method for adaptive CS-AF area selection, which extracts the magnitude-significant AF samples through a clustering approach using the density-based spatial clustering algorithm. Moreover, an appropriate criterion for the performance of the method is formalized, i.e., component concentration and preservation, as well as interference suppression, are measured utilizing the information obtained from the short-term and the narrow-band Rényi entropies, while component connectivity is evaluated using the number of regions with continuously-connected samples. The CS-AF area selection and reconstruction algorithm parameters are optimized using an automatic multi-objective meta-heuristic optimization method, minimizing the here-proposed combination of measures as objective functions. Consistent improvement in CS-AF area selection and TFD reconstruction performance has been achieved without requiring a priori knowledge of the input signal for multiple reconstruction algorithms. This was demonstrated for both noisy synthetic and real-life signals.

Fast conformational clustering of extensive molecular dynamics simulation data.

We present an unsupervised data processing workflow that is specifically designed to obtain a fast conformational clustering of long molecular dynamics simulation trajectories. In this approach, we combine two dimensionality reduction algorithms (cc_analysis and encodermap) with a density-based spatial clustering algorithm (hierarchical density-based spatial clustering of applications with noise). The proposed scheme benefits from the strengths of the three algorithms while avoiding most of the drawbacks of the individual methods. Here, the cc_analysis algorithm is applied for the first time to molecular simulation data. The encodermap algorithm complements cc_analysis by providing an efficient way to process and assign large amounts of data to clusters. The main goal of the procedure is to maximize the number of assigned frames of a given trajectory while keeping a clear conformational identity of the clusters that are found. In practice, we achieve this by using an iterative clustering approach and a tunable root-mean-square-deviation-based criterion in the final cluster assignment. This allows us to find clusters of different densities and different degrees of structural identity. With the help of four protein systems, we illustrate the capability and performance of this clustering workflow: wild-type and thermostable mutant of the Trp-cage protein (TC5b and TC10b), NTL9, and Protein B. Each of these test systems poses their individual challenges to the scheme, which, in total, give a nice overview of the advantages and potential difficulties that can arise when using the proposed method.

AISAR: Artificial Intelligence-Based Student Assessment and Recommendation System for E-Learning in Big Data

Educational systems have advanced with the use of electronic learning (e-learning), which is a promising solution for long-distance learners. Students who engage in e-learning can access tests and exams online, making education more flexible and accessible. This work reports on the design of an e-learning system that makes recommendations to students to improve their learning. This artificial intelligence-based student assessment and recommendation (AISAR) system consists of score estimation, clustering, performance prediction, and recommendation. In addition, the importance of student authentication is recognised in situations in which students must authenticate themselves prior to using the e-learning system using their identity, password, and personal identification number. Individual scores are determined using a recurrent neural network (RNN) based on student engagement and examination scores. Then, a density-based spatial clustering algorithm (DBSCAN) using Mahalanobis distance clustering is implemented to group students based on their obtained score values. The constructed clusters are validated by estimating purity and entropy. Student performance is predicted using a threshold-based MapReduce (TMR) procedure from the score-based cluster. When predicting student performance, students are classified into two groups: average and poor, with the former being divided into below- and above-average students and the latter into poor and very poor students. This categorisation aims to provide useful recommendations for learning. A recommendation reinforcement learning algorithm, the rule-based state–action–reward–state–action (R-SARSA) algorithm, is incorporated for evaluation. Students were required to work on their subjects according to the provided recommendations. This e-learning recommendation system achieves better performance in terms of true-positives, false-positives, true-negatives, false-negatives, precision, recall, and accuracy.

6P Urine spectroscopy coupled with artificial intelligence: Proof of concept for a new diagnostic tool to detect gynaecological cancers

6P Urine spectroscopy coupled with artificial intelligence: Proof of concept for a new diagnostic tool to detect gynaecological cancers

Fire Risk Assessment and Warning Based on Hierarchical Density-Based Spatial Clustering Algorithm and Grey Relational Analysis

Understanding the spatial and temporal distribution patterns of fire is of ecological, social and economic importance. The purpose of this study is to examine the spatial distribution of high fire risk using machine learning algorithms and early warning weather in high-risk areas. Take the satellite monitored fire point data in Yunnan Province during 2015–2019 as an example. The spatial distribution law of high-density parts is found using hierarchical density-based spatial clustering of applications with noise (HDBSCAN) algorithm, and the correlation degree analysis of high-density fire areas based on clustering and annual mean meteorological factors is carried out using grey correlation analysis (GCA) method. Results illustrates that within five years, fires frequently occurred in the seven regions of Wenshan Zhuang and Miao Autonomous Prefecture, Honghe Hani and Yi Autonomous Prefecture, Lijiang City, Pu’er City, and Xishuangbanna Dai Autonomous Prefecture. Among them, the fire in Lijiang had the greatest relationship with precipitation, Pu’er and Xishuangbanna had the greatest correlation with temperature, and Honghe Hani and Wenshan Zhuang were most affected by wind speed. This article acclaims fire prevention in key periods, key areas, key weather and reinforces the protection of transmission lines in the risk area.

Identification of low-resistivity-low-contrast pay zones in the feature space with a multi-layer perceptron based on conventional well log data

In the early exploration of many oilfields, low-resistivity-low-contrast (LRLC) pay zones are easily overlooked due to the resistivity similarity to the water zones. Existing identification methods are model-driven and cannot yield satisfactory results when the causes of LRLC pay zones are complicated. In this study, after analyzing a large number of core samples, main causes of LRLC pay zones in the study area are discerned, which include complex distribution of formation water salinity, high irreducible water saturation due to micropores, and high shale volume. Moreover, different oil testing layers may have different causes of LRLC pay zones. As a result, in addition to the well log data of oil testing layers, well log data of adjacent shale layers are also added to the original dataset as reference data. The density-based spatial clustering algorithm with noise (DBSCAN) is used to cluster the original dataset into 49 clusters. A new dataset is ultimately projected into a feature space with 49 dimensions. The new dataset and oil testing results are respectively treated as input and output to train the multi-layer perceptron (MLP). A total of 3192 samples are used for stratified 8-fold cross-validation, and the accuracy of the MLP is found to be 85.53%.

A convolutional neural network for segmentation of yeast cells without manual training annotations.

MotivationSingle-cell time-lapse microscopy is a ubiquitous tool for studying the dynamics of complex cellular processes. While imaging can be automated to generate very large volumes of data, the processing of the resulting movies to extract high-quality single-cell information remains a challenging task. The development of software tools that automatically identify and track cells is essential for realizing the full potential of time-lapse microscopy data. Convolutional neural networks (CNNs) are ideally suited for such applications, but require great amounts of manually annotated data for training, a time-consuming and tedious process.ResultsWe developed a new approach to CNN training for yeast cell segmentation based on synthetic data and present (i) a software tool for the generation of synthetic images mimicking brightfield images of budding yeast cells and (ii) a convolutional neural network (Mask R-CNN) for yeast segmentation that was trained on a fully synthetic dataset. The Mask R-CNN performed excellently on segmenting actual microscopy images of budding yeast cells, and a density-based spatial clustering algorithm (DBSCAN) was able to track the detected cells across the frames of microscopy movies. Our synthetic data creation tool completely bypassed the laborious generation of manually annotated training datasets, and can be easily adjusted to produce images with many different features. The incorporation of synthetic data creation into the development pipeline of CNN-based tools for budding yeast microscopy is a critical step toward the generation of more powerful, widely applicable and user-friendly image processing tools for this microorganism.Availability and implementationThe synthetic data generation code can be found at https://github.com/prhbrt/synthetic-yeast-cells. The Mask R-CNN as well as the tuning and benchmarking scripts can be found at https://github.com/ymzayek/yeastcells-detection-maskrcnn. We also provide Google Colab scripts that reproduce all the results of this work.Supplementary information Supplementary data are available at Bioinformatics online.

Application of a density-based spatial clustering algorithm for fully automatic picking of surface-wave dispersion curves

Surface waves can be used to enhance the characterization of the shallow subsurface in desert environments. A high-resolution shear-wave velocity model is typically obtained by inverting dispersion curves, which correspond to different propagation modes of the surface waves. A common approach to estimate the dispersion curves is to manually pick the magnitude maxima from the frequency-phase velocity spectra of the seismic data. This approach is inefficient, time consuming, highly subjective, and not feasible for large surveys. Automatic picking of dispersion curves has become a topic of interest recently in the oil and gas research community, where many of the developed algorithms were inherited from the fields of image processing and machine learning. By exploring in the area of unsupervised learning, we recently derived an algorithm and workflow for fully automatic picking of surface-wave dispersion curves by employing a density-based spatial clustering technique. Our approach has been tested on the SEG Advanced Modeling Corporation Arid model synthetic data set and a field data set acquired in a desert environment. The results of the synthetic tests show that the estimated dispersion curves match the true dispersion curves with high accuracy, and they can be inverted for shear-wave velocities, successfully recovering the shallow near-surface features. The application of the method to field data provides high-resolution geology-consistent shear-wave velocity information that can be converted into a compressional-wave velocity model in agreement with uphole observations.

Modification of a Density-Based Spatial Clustering Algorithm for Applications with Noise for Data Reduction in Intrusion Detection Systems

Modification of a Density-Based Spatial Clustering Algorithm for Applications with Noise for Data Reduction in Intrusion Detection Systems

Measurement-driven sequential random sample consensus GM-PHD filter for ballistic target tracking

In this paper, a new filter named measurement-driven sequential random sample consensus Gaussian mixture probability hypothesis density (MD-S-RANSAC-GM-PHD) filter is proposed for estimating the trajectory of a ballistic target during its coast phase. Unlike the traditional multiple-target tracking (MTT) algorithms that require data association, the proposed method involves modelling the respective collections of targets and measurements as random finite sets (RFS) and applying the PHD recursion to propagate the posterior intensity in time. To generate the new birth target intensity adaptively, a measurement-driven birth intensity estimation algorithm is developed. Since the measurement set used for birth intensity estimation may contain a large amount of clutter, a measurement set pre-processing method based on density-based spatial clustering and sequential random sample consensus (S-RANSAC) algorithm is proposed to eliminate the interference of clutter on generating new target birth intensity. Specifically, the proposed filter extends the standard GM-PHD filter by distinguishing between the persistent and the newborn target, and the extended Kalman filter (EKF) implementation of our proposed filter for ballistic target tracking is also derived. Simulation results illustrate the advantages of our proposed filter in tracking ballistic missile.

Unification of firefly algorithm with density-based spatial clustering for segmentation of medical images

This paper proposes a computer-aided approach for brain image segmentation to figure out various characteristics of digital images which are responsible for the identification of brain tumour with MRI images. The proposed Density-Based Spatial Clustering Fused with Firefly (DB-FF) method is based on Density-Based Spatial Clustering and Firefly Algorithm which has a significant place in nature-inspired computing techniques. In this research, the solutions of the firefly algorithm have been improved by the density-based spatial clustering algorithm and a soft computing criterion has also been used as a fitness function. The proposed method has been tested on commonly used images from Harvard Whole Brain Atlas and the results of this method have been compared with other standard benchmarks from the survey. The proposed DB-FF method achieved better segmentation than standard segmentation quality metrics such as normalised peak signal to noise, normalised root square mean error and structural similarity index metric. Matlab has been used for implementation and observation. The result demonstrates that the proposed method has a better and robust performance as compared with the existing MRI segmentation models.

An environmental dependence of the physical and structural properties in the Hydra cluster galaxies

ABSTRACT The nearby Hydra cluster (∼50 Mpc) is an ideal laboratory to understand, in detail, the influence of the environment on the morphology and quenching of galaxies in dense environments. We study the Hydra cluster galaxies in the inner regions (1R200) of the cluster using data from the Southern Photometric Local Universe Survey, which uses 12 narrow and broad-band filters in the visible region of the spectrum. We analyse structural (Sérsic index, effective radius) and physical (colours, stellar masses, and star formation rates) properties. Based on this analysis, we find that ∼88 per cent of the Hydra cluster galaxies are quenched. Using the Dressler–Schectman test approach, we also find that the cluster shows possible substructures. Our analysis of the phase-space diagram together with density-based spatial clustering algorithm indicates that Hydra shows an additional substructure that appears to be in front of the cluster centre, which is still falling into it. Our results, thus, suggest that the Hydra cluster might not be relaxed. We analyse the median Sérsic index as a function of wavelength and find that for red [(u − r) ≥2.3] and early-type galaxies it displays a slight increase towards redder filters (13 and 18 per cent, for red and early type, respectively), whereas for blue + green [(u − r)<2.3] galaxies it remains constant. Late-type galaxies show a small decrease of the median Sérsic index towards redder filters. Also, the Sérsic index of galaxies, and thus their structural properties, do not significantly vary as a function of clustercentric distance and density within the cluster; and this is the case regardless of the filter.

A Parameter for Quantifying the Macroscale Asymmetry of Tropical Cyclone Cloud Clusters

AbstractA parameter to quantify macroscale (i.e., systemwide) asymmetry of tropical cyclones (TC) in infrared satellite images, galaxy asymmetry (GASYM), which is adopted from astronomy, is described. In addition, an alternative approach to identify TC cloud clusters that is based on a density-based spatial clustering algorithm, cluster identification (CI), is presented in this study. Although a commonly used approach in TC study, the predefined radius of calculation (ROC), can be used to identify the TC region in the calculation of GASYM, this approach is not optimal because the size of the TC cloud cluster is often unknown in the calculation. The area specified by the ROC often includes pixels that do not belong to the TC cloud cluster and excludes pixels that belong to the TC cloud cluster. The CI approach addresses this issue by identifying TC cloud clusters of any size with any shape, because it depends solely on the threshold brightness temperature that corresponds to the upper bound of the brightness temperature of the specific cloud types. This study shows that the CI approach can be integrated into the GASYM calculation as an objective measure of TC symmetry. Although GASYM-CI and intensity are correlated, the relationship between GASYM-CI and intensity depends on the size of the TC cloud cluster. Comparison between GASYM and an existing objective method to quantify symmetry of TCs, the deviation angle variance technique, is also presented.

Orbit determination for fuel station in multiple SSO spacecraft refueling considering the J2 perturbation

This paper investigates the orbit determination problem for fuel station (FS) in multiple Sun-Synchronous Orbit (SSO) spacecrafts refueling mission. A novel K-M-N transportation network to solve this general refueling problem is proposed, which contains several FSs with their reusable servicing spacecraft (SSC) and a large number of targets. The proposed network allows the FS drifting in a cooperative rendezvous to reduce the fuel consumption considering the J2 perturbation. Furthermore, one-to-one mode is chosen as the refueling strategy between the FS and the target, which makes it feasible to determine the number and orbit of the FS. In order to achieve such an economical refueling strategy, the problem is formulated as a facility location–allocation (FLA) model, subject to the different demands among targets and the limitation of SSC's capacity. Then, a hybrid Density-Based Spatial clustering algorithm (HDBSCAN) is designed to solve the multi-objective optimization model. Finally, two numerical examples are presented to illustrate the effectiveness and validity of the proposed algorithm.

Leveraging Artificial Intelligence for Large-Scale Plant Phenology Studies From Noisy Time-Lapse Images

Phenology has become a field of growing importance due to the increasingly apparent impacts of climate change. However, the time-consuming, subjective and tedious nature of traditional human field observations have hindered the development of large-scale phenology networks. Such networks are rare and rely on time-lapse cameras and simplistic color indexes to monitor phenology. To automatize rapid, detailed and repeatable analyzes, we propose an Artificial Intelligence (AI) framework based on machine learning and computer vision techniques. Our approach extracts multiple ecologically-relevant indicators from time-lapse digital photography datasets. The proposed framework consists of three main components: (i) a random forest model to automatically select relevant images based on color information; (ii) a convolutional neural network (CNN) to identify and localize open tree buds; and (iii) a density-based spatial clustering algorithm to cluster open bud detections across the time-series. We tested this framework on a dataset including thousands of black spruce and balsam fir tree images captured using our phenological camera network. The performed experiments showed the efficiency of the proposed approach under challenging perturbation factors, such as significant image noise. Our framework is exceedingly faster and more accurate than human analysts, reducing the time-series processing time from multiple days to under an hour. The proposed methodology is particularly appropriate for large-scale and long-term analyzes of ecological imagery datasets. Our work demonstrates that the use of computer vision and machine learning methods represents a promising direction for the implementation of national, continental, or even global plant phenology networks.

Adaptive Density-Based Spatial Clustering for Massive Data Analysis

Clustering is a classical research field due to its broad applications in data mining such as emotion detection, event extraction and topic discovery. It aims to discover intrinsic patterns which can be formed as clusters from a collection of data. Significant progress have been made by the Density-based Spatial Clustering of Applications with Noise (DBSCAN) and its variants. However, there is a major limitation that current density-based algorithms suffer from linear connection problem, where they perform poorly to discriminate objective clusters which are “connected” by a few data points. Moreover, the parameter setting and the time cost make it hard to be well-adapted in massive data analysis. To address these problems, we propose a novel adaptive density-based spatial clustering algorithm called Ada-DBSCAN, which consists of a data block splitter and a data block merger, coordinated by local clustering and global clustering. We conduct extensive experiments on both artificial and real-world datasets to evaluate the effectiveness of Ada-DBSCAN. Experimental results show that our algorithm evidently outperforms several strong baselines in both clustering accuracy and human evaluation. Besides, Ada-DBSCAN shows significant improvement of efficiency compared with DBSCAN.