Real-time Clustering Research Articles

Clustering bike sharing stations using Quantum Machine Learning: A case study of Toronto, Canada

Quantum Machine Learning (QML) is a field that combines the principles of Quantum Computing (QC) and Machine Learning (ML). QC works by taking advantage of the properties of quantum physics, such as superposition and entanglement. To fully realize the potential of this technology, more research is necessary, as the field of QML is still in its early stages. Since QC technologies and devices continue to develop quickly, it is important to identify the use cases and applications that benefit the most. This paper investigates the potentials of QC, and more specifically, Quantum Annealing (QA), for clustering real-world data in transportation systems. The Bike Sharing System (BSS) is used as a case study applying a clustering model on QA computers. The main contribution of this research is to introduce a hybrid model to cluster stations in a BSS by solving it as a Constraint Satisfaction Problem (CSP) problem with different methods on a QA computer using a real-time dataset. In addition to the practical contribution, this research also offers theoretical advancements in the field of computational optimization by defining a new topology for the input data that is compatible with QC topology (e.g., Chimera topology). The goal of real-time clustering BSS stations based on dynamic and static datasets is, in fact, to assist decision-makers in better managing and minimizing the risk of bike unavailability at each station and rebalancing bikes shared. Three different methods have been used to determine the number of clusters, and Euclidean, Manhattan, Pearson, and Spearman dissimilarity functions have been applied to cluster the stations. The evaluation is done using the magnitude vs. cardinality approach. The distribution of the stations, magnitude, and cardinality of the results indicate the potential to use QC for clustering for a real-world application, e.g., BSS.

Analysing social media with an Improved K-Means Clustering Algorithm

Nowadays, sharing human social behavior and growing a multi-user network requires the use of social media. Analysis of social media provides a valuable opportunity to study human social activity at scale. People often bring and talk about different kind of topics on social media platforms. depending on which, discussions are made to show the positive as well dis positives ways. This is an interesting point about using social media. There is a great deal of information about the type and substance of these discussion holography’s that informs us more broadly around patterns in social media interactions, and how content flows between individuals. Data mining techniques exists which can be utilized to crunch out user information from social media and relationships existing within the network. However, state of the art methods often fail to model user communities and their behaviors at a correct level. To solve this problem and help successfully do the grouping of related information, in our work here we present an enhanced fuzzy means clustering approach. Using this technique reduces the creation and time complexity of clusters, resulting in higher quality group outcomes. Our proposed system aims at mitigating the problems as described above by introducing a scalable and effective solution for real-time cluster social media data.

Real-Time Point Cloud Clustering Algorithm Based on Roadside LiDAR

Real-Time Point Cloud Clustering Algorithm Based on Roadside LiDAR

Monitoring Dynamically Changing Migratory Flocks Using an Algebraic Graph Theory-Based Clustering Algorithm

Migration flocks have different forms, including single individuals, formations, and irregular clusters. The shape of a flock can change swiftly over time. The real-time clustering of multiple groups with different characteristics is crucial for the monitoring of dynamically changing migratory flocks. Traditional clustering algorithms need to set various prior parameters, including the number of groups, the number of nearest neighbors, or the minimum number of individuals. However, flocks may display complex group behaviors (splitting, combination, etc.), which complicate the choice and adjustment of the parameters. This paper uses a real-time clustering-based method that utilizes concepts from the algebraic graph theory. The connected graph is used to describe the spatial relationship between the targets. The similarity matrix is calculated, and the problem of group clustering is equivalent to the extraction of the partitioned matrices within. This method needs only one prior parameter (the similarity distance) and is adaptive to the group’s splitting and combination. Two modifications are proposed to reduce the computation burden. First, the similarity distance can be broadened to reduce the exponent of the similarity matrix. Second, the omni-directional measurements are divided into multiple sectors to reduce the dimension of the similarity matrix. Finally, the effectiveness of the proposed method is verified using the experimental results using real radar data.

Two-step spatiotemporal anomaly detection corrected for lag reporting time with application to real-time dengue surveillance in Thailand

BackgroundDengue infection ranges from asymptomatic to severe and life-threatening, with no specific treatment available. Vector control is crucial for interrupting its transmission cycle. Accurate estimation of outbreak timing and location is essential for efficient resource allocation. Timely and reliable notification systems are necessary to monitor dengue incidence, including spatial and temporal distributions, to detect outbreaks promptly and implement effective control measures.MethodsWe proposed an integrated two-step methodology for real-time spatiotemporal cluster detection, accounting for reporting delays. In the first step, we employed space-time nowcasting modeling to compensate for lags in the reporting system. Subsequently, anomaly detection methods were applied to assess adverse risks. To illustrate the effectiveness of these detection methods, we conducted a case study using weekly dengue surveillance data from Thailand.ResultsThe developed methodology demonstrated robust surveillance effectiveness. By combining space-time nowcasting modeling and anomaly detection, we achieved enhanced detection capabilities, accounting for reporting delays and identifying clusters of elevated risk in real-time. The case study in Thailand showcased the practical application of our methodology, enabling timely initiation of disease control activities.ConclusionOur integrated two-step methodology provides a valuable approach for real-time spatiotemporal cluster detection in dengue surveillance. By addressing reporting delays and incorporating anomaly detection, it complements existing surveillance systems and forecasting efforts. Implementing this methodology can facilitate the timely initiation of disease control activities, contributing to more effective prevention and control strategies for dengue in Thailand and potentially other regions facing similar challenges.

Practical Challenges in Nanofluid Convective Heat Transfer Inside Silicon Microchannels

Despite numerous studies on nanofluids in microchannel heat sinks (MCHSs), they are not yet commercialized due to long-term stability issues and high maintenance costs. Therefore, this study explores the impact of nanofluids and nanoparticle clustering on single-phase convective heat transfer inside microchannels under laminar conditions. Water and commercially available water-based nanosuspensions, including Al2O3-water (30–60 nm), TiO2-water (5–30 nm), and polystyrene-water (50 nm), are circulated through silicon MCHS having rectangular channels integrated into a closed flow loop. To assess the in situ and real-time nanoparticle clustering during heat transfer experiments, Light Extinction Spectroscopy (LES) is applied as a non-intrusive measurement technique on nanofluids without any fluid sampling. Our findings reveal the appearance of nanofluid discoloration with no measurable increase in heat transfer coefficient. This unexpected change is attributed to the interplay of abrasion, erosion, and corrosion phenomena, likely triggered by the clustering of nanoparticles within the silicon microchannels—a novel insight into the complex dynamics of nanofluid behavior (an increase in the De Brouckere mean diameter from 11 nm to 107.3 nm over a 2.5 h period for TiO2 nanoparticles). The resulting material loss could not be mitigated by altering the nanoparticle material, which may impede heat transfer enhancement under tested conditions.

Reduced Scaling Real-Time Coupled Cluster Theory.

Real-time coupled cluster (CC) methods have several advantages over their frequency-domain counterparts, namely, response and equation of motion CC theories. Broadband spectra, strong fields, and pulse manipulation allow for the simulation of complex spectroscopies that are unreachable using frequency-domain approaches. Due to the high-order polynomial scaling, the required numerical time propagation of the CC residual expressions is a computationally demanding process. This scaling may be reduced by local correlation schemes, which aim to reduce the size of the (virtual) orbital space by truncation according to user-defined parameters. We present the first application of local correlation to real-time CC. As in previous studies of locally correlated frequency-domain CC, traditional local correlation schemes are of limited utility for field-dependent properties; however, a perturbation-aware scheme proves promising. A detailed analysis of the amplitude dynamics suggests that the main challenge is a strong time dependence of the wave function sparsity.

An autocorrelation incremental fuzzy clustering framework based on dynamic conditional scoring model

This paper focuses on the real-time dynamic clustering analysis of power load data based on the dynamic conditional score (DCS) model, and an autocorrelation increment fuzzy C-means clustering algorithm based on the DCS model is proposed. (1) The paper addresses the problem that current power load clustering methods, when performing time series data mining, tend to focus on capturing the mean structure while ignoring the variance characteristics of the data, making it difficult to effectively capture the structural information of time series data. The DCS model is used as the statistical model basis for clustering analysis, and the time series is clustered based on the estimated conditional moment characteristics of the model, dynamically capturing data features such as the mean, variance, and sequence correlation of time series data, effectively improving the clustering performance. (2) This paper also addresses the issue that current power load clustering methods tend to focus on static datasets of user power loads and cannot effectively handle the data stream clustering problem with time series characteristics in practical applications. The DCS model parameter dataset and the autocorrelation increment fuzzy clustering algorithm are used to conduct a dynamic data flow analysis of user electricity behaviour evolution and pattern continuous updating research for power loads. The clustering results are dynamically updated based on the user's power load data stream using the proposed algorithm, achieving research on a universal clustering model and secure and efficient algorithms in a big data environment. (3) The paper verifies the clustering performance of the proposed method using power load time series data provided by a Chinese power supply company as a case dataset. The clustering evaluation index shows that the proposed algorithm has high clustering accuracy and good clustering performance. Additionally, different power supply recommendations are proposed for different customer electricity types in the obtained clustering results to provide more personalized power services.

A FPGA-Based Architecture for Real-Time Cluster Finding in the LHCb Silicon Pixel Detector

This article describes a custom VHDL firmware implementation of a two-dimensional cluster-finder architecture for reconstructing hit positions in the new vertex pixel detector (VELO) that is part of the LHCb Upgrade. This firmware has been deployed to the existing FPGA cards that perform the readout of the VELO, as a further enhancement of the DAQ system, and will run in real time during physics data taking, reconstructing VELO hits coordinates on-the-fly at the LHC collision rate. This pre-processing allows the first level of the software trigger to accept a 11% higher rate of events, as the ready-made hits coordinates accelerate the track reconstruction and consumes significantly less electrical power. It additionally allows the raw pixel data to be dropped at the readout level, thus saving approximately 14% of the DAQ bandwidth. Detailed simulation studies have shown that the use of this real-time cluster finding does not introduce any appreciable degradation in the tracking performance in comparison to a full-fledged software implementation. This work is part of a wider effort aimed at boosting the real-time processing capability of HEP experiments by delegating intensive tasks to dedicated computing accelerators deployed at the earliest stages of the data acquisition chain.

Real-Time Framework to Predict Crash Likelihood and Cluster Crash Severity

This study proposes a three-stage framework for real-time crash likelihood and severity prediction. Firstly, a real-time crash likelihood prediction model was developed. Secondly, a real-time crash severity clustering model was proposed to cluster the crashes into different severity levels. Thirdly, a severity clustering validation model was developed to assess the performance of the proposed severity clustering model. Extensive data processing techniques were employed to collect real-time features from State Road 408 in Orlando, Florida, and a total of 6,750,072 events (625 crash events and 6,749,447 non-crash events) along with 24 real-time features were used. To develop the crash likelihood prediction model, nine machine-learning techniques were attempted, and the convolutional neural network model was found to provide the best result with respect to the sensitivity (0.916), false alarm rate (0.111), and area under the receiver operating characteristic curve (0.967). Davies–Bouldin index criteria were used to find the detector location that generated the most accurate traffic information to cluster the crashes into severity levels, and based on this traffic information, k-means clustering was applied to develop the severity clustering model. Finally, a severity clustering validation model was developed after investigating nine machine-learning techniques to validate the developed severity clustering model, and the decision tree model provided the best results based on three levels of sensitivity and specificity values. The developed framework has the potential to help traffic management centers to warn road users or develop transportation systems management and operations strategies in real time to avoid crashes or minimize the severity and, thus, can significantly contribute to improving road safety.

A new scheme of plasma control system based on real-time Linux cluster for HL-2M

HL-2M is a medium-sized tokamak constructed by the Southwestern Institute of Physics (SWIP) in China. A new plasma control system (PCS) based on the software framework of DIII-D PCS for HL-2M has been proposed to replace the original loose system. The new hardware system proposed in this paper consists of three computer nodes and peripheral equipment. Data acquisition is completed by a standing server and a reflective memory network that passes back data. The software architecture of DIII-D PCS is generally retained, but some interfaces are added to adapt to the existing system. In addition, the Linux operating system has also been upgraded and optimized so that real-time tasks can achieve better performance. The initial system involves some basic control algorithms satisfying early experiments. The new PCS has been preliminarily verified through simulation and can output control commands as expected in the integrated environment. This paper focuses on two aspects of DIII-D PCS on a new device, i.e., performance and interface adaptation. More efforts will be made to add more algorithms and use the system in real experiments.

Real-time remaining useful life kernel density estimation considering dynamic transition of degradation states

The accurate prediction of remaining useful life is a significant issue for ensuring the reliable operation of the system. Considering the dynamic transfer of degenerate state can improve the prediction accuracy and reduce the number of late prediction. Firstly, a time series density peak clustering algorithm suitable for real-time manifold data clustering is proposed. By choosing larger truncation distance at points with high sample density, the cluster centers can be found more accurately. Then, different degradation state patterns according to clustering results can be divided. Moreover, the smoothing parameters can be adaptively updated according to the sample density under different degradation modes and an adaptive kernel density remaining useful life estimation model is established. The test of the gearbox verifies the necessity and accuracy of the proposed model by comparison with the remaining useful life predictions of kernel density estimation without considering degraded state transitions.

Power Big Data Analysis Platform Design Based on Hadoop

To better analyze and process power data to obtain effective information, a power big data analysis scheme based on Hadoop architecture is proposed. We analyze the cloud computing environment Hadoop distributed platform to obtain massive data of large-scale distributed power system. According to the characteristics of intelligent power consumption data, common data mining algorithm modules such as parallel classification algorithm and parallel real-time clustering algorithm are designed, and the implementation of clustering algorithm with different principles is further analyzed. Then, HBase is adopted to access data in a distributed way, and MapReduce is used to realize the visual management of power GIS data. The experimental results show that the parallel processing method of power big data based on Hadoop has high efficiency and good scalability, and the algorithm has good identification ability for massive data in the cluster mode.

Ending the HIV Epidemic: Identifying Barriers and Facilitators to Implement Molecular HIV Surveillance to Develop Real-Time Cluster Detection and Response Interventions for Local Communities.

The rapid implementation of molecular HIV surveillance (MHS) has resulted in significant challenges for local health departments to develop real-time cluster detection and response (CDR) interventions for priority populations impacted by HIV. This study is among the first to explore professionals' strategies to implement MHS and develop CDR interventions in real-world public health settings. Methods: Semi-structured qualitative interviews were completed by 21 public health stakeholders in the United States' southern and midwestern regions throughout 2020-2022 to identify themes related to the implementation and development of MHS and CDR. Results for the thematic analysis revealed (1) strengths and limitations in utilizing HIV surveillance data for real-time CDR; (2) limitations of MHS data due to medical provider and staff concerns related to CDR; (3) divergent perspectives on the effectiveness of partner services; (4) optimism, but reluctance about the social network strategy; and (5) enhanced partnerships with community stakeholders to address MHS-related concerns. Conclusions: Enhancing MHS and CDR efforts requires a centralized system for staff to access public health data from multiple databases to develop CDR interventions; designating staff dedicated to CDR interventions; and establishing equitable meaningful partnerships with local community stakeholders to address MHS concerns and develop culturally informed CDR interventions.

Study of cell-cell adhesion formation and dynamics in cancer cells using a hybrid cell-lipid bilayer system.

The formation and regulation of connections between cells play a crucial role in maintaining tissue homeostasis and tumour repression. In many tissues, particular importance comes to adherens junctions that are formed by the binding of E-cadherin proteins at the surface of adjacent cells, followed by their clustering and recruitment of catenins and other adaptor proteins on the intracellular site linking the protein complex to the actomyosin cytoskeleton. This process depends on multiple parameters such as the mobility, density, type of cadherins, and the mechanical forces acting on the cadherin-catenin complexes and can change dramatically during developmental stages or in diseases such as cancer. To study the initial steps of adherens junction formation in a controlled environment in cancer cells, we have established a reconstituted system of cells adhering to supported lipid bilayers decorated with extracellular domains of E-cadherin proteins. This reconstituted system allowed us to capture the initiation and dynamics of cadherin interactions, the real-time localisation and clustering of adaptor proteins and the maturation of adherens junction in cancer cells over time using live cell imaging with confocal and total internal reflection microscopy. To understand the importance of the different parameters and to mimic the cell/tissue abnormalities occurring in cancer scenarios, we then perturb the controlled system by varying cadherin density, membrane mobility and inducing mechanical stress by confining the cell system over time. We find, for example, that reduced E-cadherin mobility leads to the formation of stable regions showing actin enrichment and that confinement increases the steady state adhesion zone. This approach provides new insights about cell surface dynamics during cell-cell adhesion and can improve our understanding of the role of cell mechanics and E-cadherin organisation patterns in cancer progression.

Energy-balanced neuro-fuzzy dynamic clustering scheme for green & sustainable IoT based smart cities

The Internet of Things (IoT) is a pervasive computing technology that provides solutions to critical sustainable smart city applications. Each sustainable application has its own set of requirements, including energy efficiency, Quality of Service (QoS), hardware, and software resources. Even though green IoT devices operate in a resource-constrained environment. Monitoring, recognizing, and responding to activities that entail continuous access to timely information in a partially or fully distributed ecosystem is a difficult task. To overcome the challenges of resource management in the IoT, we proposed an energy-efficient Dynamic Clustering Routing (DCR) protocol using a neuro-fuzzy technique for restricting the resources of IoT devices. The proposed protocol uses a dynamic self-organizing neural network to create dynamic clusters in a network. The test-bed analysis is for computing the real-time event detection and clustering sensor nodes using TinyOS. The simulation result shows that the proposed protocol achieved a significant gain over peer-competing well-known green communication routing protocols like Low-energy Adaptive Clustering Hierarchy (LEACH) and Low-energy Adaptive Clustering Hierarchy-Centralized (LEACH-C). The proposed model results show that using neuro-fuzzy logic is effective for sustainable IoT devices and green smart city applications in terms of resource management and dynamic clustering. The result analysis shows that the proposed protocol shows an average 35% significant gain on the First Node Dies (FND), Last Node Dies (LND), the number of packets sent to CH & BS, network convergence time, network overhead, and average packet delay to compare with the LEACH and LEACH-C.

A novel multi-viewpoints based cosine similarity visual technique for an effective assessment of clustering tendency

Data clustering is an unsupervised technique that can be used to partition the data into groups based on the similarities of the retrieved objects using different distance metrics like Euclidean, cosine, etc. In contrast to Euclidean, the cosine computes the object's similarity by considering both the magnitude and direction of the data vectors. As a result, it performed far better than a standard Euclidean distance metric in applications involving real-time data clustering. The initial k-value (clustering tendency) is required by top clustering techniques like k-means and hierarchical approaches to determine the clusters' quality. Users with knowledge can assign the k-value. However, sometimes the right k-value in such algorithms may need to be assigned. After a thorough review of the work, it was discovered that the visual technique known as visual assessment of (cluster) tendency (VAT) effectively addresses the clustering tendency issue. It uses the Euclidean metric to find the similarity features in its algorithm. Another enhanced visual technique, cosinebased VAT (cVAT), outperformed the VAT for text data and speech clustering applications. However, the similarity features are extracted about a single viewpoint in cVAT. This paper develops the multi-viewpoints-based cosine similarity measure (MVPCSM) for a more informative assessment. Instead of using a single reference point like a typical cosine measure, the MVPCSM generates precise similarity characteristics using several views. The performance of the existing and proposed technique (MVPCSM-VAT) is evaluated using clustering accuracy (CA) and normalized mutual information (NMI). It has been demonstrated that the proposed MVPCSM-VAT is 15-25% more efficient than VAT and cVAT in terms of the parameters of CA and NMI. The proposed method successfully obtains more quality data clusters than MVS-VAT.

Real-Time Clustering Based on Deep Embeddings for Threat Detection in 6G Networks

Real-Time Clustering Based on Deep Embeddings for Threat Detection in 6G Networks

Large-Scale Distributed System and Design Methodology for Real-Time Cluster Services and Environments

The demand for a large-scale distributed system, such as a smart grid, which includes real-time interconnection, is rapidly increasing. To provide a seamless connected environment, real-time communication and optimal resource allocation of cluster microgrid platforms (CMPs) are essential. In this paper, we propose two techniques for real-time interconnection and optimal resource allocation for a large-scale distributed system. In particular, to configure a CMP, we analyze the data transfer rate and utilization rate from the intelligent electronic device (IED), collecting the power production data to the individual controller. The details provided in this paper are used to design a sample value, i.e., raw data transfer, on the basis of the IEC 61850 protocol for mapping. The choice of sampled values is to attain the critical time requirement, data transmission of current transformers, voltage transformers, and protective relaying of less than 1 s without complicating the real-time implementation. Furthermore, in this paper, a way to determine the optimal number of physical resources (i.e., CPU, memory, and network) for a given system is discussed. CPU ranged from 0.9 to 0.98 while each cluster increased from 10 to 1000. With the same condition, memory utilized almost 100% utilization from 0.98 to 1. Lastly, the network utilization rate was 0.96 and peaked at 1 at most. Based on the results, we confirm that a large-scale distributed system can provide a seamless monitoring service to distribute messages for each IED, and this can provide a configuration for CMP without exceeding 100% utilization.

Luminosity and beam-induced background measurement with the CMS Tracker Endcap Pixel Detector at HL-LHC

The High-Luminosity upgrade of the LHC (HL-LHC) places unprecedented requirements for background monitoring and luminosity measurements. The CMS Tracker Endcap Pixel Detector (TEPX) will be adapted to provide high-precision online measurements of bunch-by-bunch luminosity and beam-induced background. The implementation of dedicated triggering and readout systems, the real-time clustering algorithm on an FPGA and the expected performance are discussed.