Locality Sensitive Hashing Technique Research Articles

Privacy-preserving sports data fusion and prediction with smart devices in distributed environment

With the rapid advancement of sports analytics and fan engagement technologies, the volume and diversity of physique data generated by smart devices across various distributed sports platforms have grown significantly. Extracting insights and enhancing fan experiences from such data offer considerable benefits. Yet, this process unveils two primary challenges. Firstly, efficiently utilizing the vast datasets in sports analytics is daunting due to the complex nature of the sports industry. Secondly, the data collected from diverse sources and stored in distributed platforms contain sensitive information like fan preferences and athlete performance metrics, posing risks of privacy breaches. To address these challenges, we leverage an advanced Locality-Sensitive Hashing technique, known as PSDFPALSH\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$_{\ ext {ALSH}}$$\\end{document}, tailored for the sports domain. This paper presents a new privacy-preserving method for sports data fusion and prediction in distributed environments, utilizing enhanced Locality-Sensitive Hashing to protect sensitive information while maintaining high data utility. Through extensive experimentation, our approach demonstrates superior performance over existing methods in terms of Mean Absolute Error (MAE), Root Mean Squared Error (RMSE), and computational efficiency.

MPSketch: Message Passing Networks via Randomized Hashing for Efficient Attributed Network Embedding.

Given a network, it is well recognized that attributed network embedding represents each node of the network in a low-dimensional space, and, thus, brings considerable benefits for numerous graph mining tasks. In practice, a diverse set of graph tasks can be processed efficiently via the compact representation that preserves content and structure information. The majority of attributed network embedding approaches, especially, the graph neural network (GNN) algorithms, are substantially costly in either time or space due to the expensive learning process, while the randomized hashing technique, locality-sensitive hashing (LSH), which does not need learning, can speedup the embedding process at the expense of losing some accuracy. In this article, we propose the MPSketch model, which bridges the performance gap between the GNN framework and the LSH framework by adopting the LSH technique to pass messages and capture high-order proximity in a larger aggregated information pool from the neighborhood. The extensive experimental results confirm that in node classification and link prediction, the proposed MPSketch algorithm enjoys performance comparable to the state-of-the-art learning-based algorithms and outperforms the existing LSH algorithms, while running faster than the GNN algorithms by 3-4 orders of magnitude. More precisely, MPSketch runs 2121, 1167, and 1155 times faster than GraphSAGE, GraphZoom, and FATNet on average, respectively.

FSPLO: a fast sensor placement location optimization method for cloud-aided inspection of smart buildings

With the awakening of health awareness, people are raising a series of health-related requirements for the buildings they live in, with a view to improving their living conditions. In this context, BIM (Building Information Modeling) makes full use of cutting-edge theories and technologies in many domains such as health, environment, and information technology to provide a new way for engineers to design and build various healthy and green buildings. Specifically, sensors are playing an important role in achieving smart building goals by monitoring the surroundings of buildings, objects and people with the help of cloud computing technology. In addition, it is necessary to quickly determine the optimal sensor placement to save energy and minimize the number of sensors for a building, which is a de-trial task for the cloud platform due to the limited number of sensors available and massive candidate locations for each sensor. In this paper, we propose a Fast Sensor Placement Location Optimization approach (FSPLO) to solve the BIM problem in cloud-aided smart buildings. In particular, we quickly filter out the repeated candidate locations of sensors in FSPLO using Locality Sensitive Hashing (LSH) techniques to maintain only a small number of optimized locations for deploying sensors around buildings. In this way, we can significantly reduce the number of sensors used for health and green buildings. Finally, a set of simulation experiments demonstrates the excellent performance of our proposed FSPLO method.

LSH-based missing value prediction for abnormal traffic sensors with privacy protection in edge computing

Traffic flow prediction is an important part of intelligent transportation systems (ITS). However, sensor failures or the transmission distortion often occur in the process of data acquisition, which will inevitably cause the loss or abnormality of traffic flow data transmitted to the edge server. In this situation, it is necessary to share traffic flow data among different platforms. However, existing traffic flow prediction methods are facing two challenges in the process of traffic flow data sharing. First, user privacy is often leaked in the process of sharing traffic data on various platforms. Moreover, with the continuous updating of data, the efficiency and scalability of data sharing between different platforms will become lower and lower. In view of the above challenges, in this paper, we propose a novel prediction method for the missing traffic flow data caused by abnormal sensors, named ASMVP_{distr-LSH} based on distributed locality-sensitive hashing (LSH) technique. At last, a case study is presented to illustrate the feasibility and effectiveness of our approach ASMVP_{distr-LSH}.

Privacy-Aware Factorization-Based Hybrid Recommendation Method for Healthcare Services

With the advancements of the Health 2.0 technology, large-scale healthcare services are available online. Recommender systems for healthcare services have emerged for decision assistance. Most existing collaborative recommendation algorithms only mine global interactions while failing to capture the local different information of users or items. Besides, privacy concern is another significant problem to be considered in recommender systems for healthcare services. In this article, a privacy-aware factorization-based hybrid method is proposed for healthcare service recommendations. For better modeling of user preferences and service features, multiview embeddings of users and healthcare services are learned. Besides, we address the privacy problem by integrating local differential privacy and locality-sensitive hashing techniques into the recommendation model for privacy-aware neighbor searching. The final prediction is made based on a stochastic gradient descent learning-based hybrid collaborative model. Experiments demonstrate the effectiveness of the proposed method in both recommendation performance and privacy concerns.

Clustering of Template-Generated Webpages Using DOM Tree Paths of URLs

Web templates are layouts for webpages that enable rapid and easy access to web content. Web data integration solutions use template based wrapper tools to extract product information from e-commerce websites. Given a collection of webpages, wrapper tools are used to discover the template portion of a webpage and extract data from it. These wrapper based data extraction techniques require pages created with the same template belong to the same cluster. Clustering these webpages based on their template is a significant challenge. While there are algorithms for clustering webpages based on their template, they are computationally intensive to be applied at web scale. By examining the DOM tree paths of URLs on a webpage, the proposed work presents a highly scalable methodology for clustering template-generated webpages. Further, the locality sensitive hashing (LSH) technique is used to reduce the cost of clustering. The proposed technique is found to be more precise and cost effective than the existing baseline methods when tested on three separate real-time data sets.

PSEUDo: Interactive Pattern Search in Multivariate Time Series with Locality-Sensitive Hashing and Relevance Feedback.

We present PSEUDo, a visual pattern retrieval tool for multivariate time series. It aims to overcome the uneconomic (re-)training problem accompanying deep learning-based methods. Very high-dimensional time series emerge on an unprecedented scale due to increasing sensor usage and data storage. Visual pattern search is one of the most frequent tasks on time series. Automatic pattern retrieval methods often suffer from inefficient training data, a lack of ground truth labels, and a discrepancy between the similarity perceived by the algorithm and required by the user or the task. Our proposal is based on the query-aware locality-sensitive hashing technique to create a representation of multivariate time series windows. It features sub-linear training and inference time with respect to data dimensions. This performance gain allows an instantaneous relevance-feedback-driven adaption to converge to users' similarity notion. We demonstrate PSEUDo's performance in terms of accuracy, speed, steerability, and usability through quantitative benchmarks with representative time series retrieval methods and a case study. We find that PSEUDo detects patterns in high-dimensional time series efficiently, improves the result with relevance feedback through feature selection, and allows an understandable as well as user-friendly retrieval process.

Time-Aware Cross-Platform IoT Service Recommendation with Privacy Preservation

IoT service recommendation techniques can help a user select appropriate IoT services efficiently. Aiming at improving the recommendation efficiency and preserving the data privacy, the locality-sensitive hashing (LSH) technique is adopted in service recommendation. However, existing LSH-based service recommendation methods ignore the intrinsic temporal feature of IoT services. In light of this challenge, we integrate the temporal feature into the conventional LSH-based method and present a time-aware approach with the capability of privacy preservation for IoT service recommendation across multiple platforms. Experiments on a real-world dataset are conducted to validate the advantage of our proposed approach in terms of accuracy and efficiency in recommendation.

An LSH-based [formula omitted]-representatives clustering method for large categorical data

Clustering categorical data remains a challenging problem in the era of big data, due to the difficulty in measuring dis/similarity meaningfully for categorical data and the high computational complexity of existing clustering algorithms that makes it difficult to be applied in practical use for big data mining applications. In this paper, we propose an integrated approach that incorporates the Locality-Sensitive Hashing (LSH) technique into the k-means-like clustering so as to make it capable of predicting the better initial clusters for boosting clustering effectiveness. To this end, we first utilize a data-driven dissimilarity measure for categorical data to construct a family of binary hash functions that are then used to generate the initial clusters. We also propose to use a nearest neighbor search at each iteration for cluster reassignment of data objects to improve the clustering complexity. These solutions are incorporated into the k-representatives algorithm resulting in the so-called LSH-k-representatives algorithm. Extensive experiments conducted on multiple real-world and synthetic datasets have demonstrated the effectiveness of the proposed method. It is shown that the newly developed algorithm yields comparable or better clustering results in comparison to the existing closely related works, yet it is significantly more efficient by a factor of between 2× and 32×.

Noise-tolerant similarity search in temporal medical data

Temporal medical data are increasingly integrated into the development of data-driven methods to deliver better healthcare. Searching such data for patterns can improve the detection of disease cases and facilitate the design of preemptive interventions. For example, specific temporal patterns could be used to recognize low-prevalence diseases, which are often under-diagnosed. However, searching these patterns in temporal medical data is challenging, as the data are often noisy, complex, and large in scale. In this work, we propose an effective and efficient solution to search for patients who exhibit conditions that resemble the input query. In our solution, we propose a similarity notion based on the Longest Common Subsequence (LCSS), which is used to measure the similarity between the query and the patient’s temporal medical data and to ensure robustness against noise in the data. Our solution adopts locality sensitive hashing techniques to address the high dimensionality of medical data, by embedding the recorded clinical events (e.g., medications and diagnosis codes) into compact signatures. To perform pattern search in large EHR datasets, we propose a filtering approach based on tandem patterns, which effectively identifies candidate matches while discarding irrelevant data. The evaluations conducted using a real-world dataset demonstrate that our solution is highly accurate while significantly accelerating the similarity search.

An effective recommender system based on personality traits, demographics and behavior of customers in time context

PurposeImproving the performance of recommender systems (RSs) has always been a major challenge in the area of e-commerce because the systems face issues such as cold start, sparsity, scalability and interest drift that affect their performance. Despite the efforts made to solve these problems, there is still no RS that can solve or reduce all the problems simultaneously. Therefore, the purpose of this study is to provide an effective and comprehensive RS to solve or reduce all of the above issues, which uses a combination of basic customer information as well as big data techniques.Design/methodology/approachThe most important steps in the proposed RS are: (1) collecting demographic and behavioral data of customers from an e-clothing store; (2) assessing customer personality traits; (3) creating a new user-item matrix based on customer/user interest; (4) calculating the similarity between customers with efficient k-nearest neighbor (EKNN) algorithm based on locality-sensitive hashing (LSH) approach and (5) defining a new similarity function based on a combination of personality traits, demographic characteristics and time-based purchasing behavior that are the key incentives for customers' purchases.FindingsThe proposed method was compared with different baselines (matrix factorization and ensemble). The results showed that the proposed method in terms of all evaluation measures led to a significant improvement in traditional collaborative filtering (CF) performance, and with a significant difference (more than 40%), performed better than all baselines. According to the results, we find that our proposed method, which uses a combination of personality information and demographics, as well as tracking the recent interests and needs of the customer with the LSH approach, helps to improve the effectiveness of the recommendations more than the baselines. This is due to the fact that this method, which uses the above information in conjunction with the LSH technique, is more effective and more accurate in solving problems of cold start, scalability, sparsity and interest drift.Research limitations/implicationsThe research data were limited to only one e-clothing store.Practical implicationsIn order to achieve an accurate and real-time RS in e-commerce, it is essential to use a combination of customer information with efficient techniques. In this regard, according to the results of the research, the use of personality traits and demographic characteristics lead to a more accurate knowledge of customers' interests and thus better identification of similar customers. Therefore, this information should be considered as a solution to reduce the problems of cold start and sparsity. Also, a better judgment can be made about customers' interests by considering their recent purchases; therefore, in order to solve the problems of interest drifts, different weights should be assigned to purchases and launch time of products/items at different times (the more recent, the more weight). Finally, the LSH technique is used to increase the RS scalability in e-commerce. In total, a combination of personality traits, demographics and customer purchasing behavior over time with the LSH technique should be used to achieve an ideal RS. Using the RS proposed in this research, it is possible to create a comfortable and enjoyable shopping experience for customers by providing real-time recommendations that match customers' preferences and can result in an increase in the profitability of e-shops.Originality/valueIn this study, by considering a combination of personality traits, demographic characteristics and time-based purchasing behavior of customers along with the LSH technique, we were able for the first time to simultaneously solve the basic problems of CF, namely cold start, scalability, sparsity and interest drift, which led to a decrease in significant errors of recommendations and an increase in the accuracy of CF. The average errors of the recommendations provided to users based on the proposed model is only about 13%, and the accuracy and compliance of these recommendations with the interests of customers is about 92%. In addition, a 40% difference between the accuracy of the proposed method and the traditional CF method has been observed. This level of accuracy in RSs is very significant and special, which is certainly welcomed by e-business owners. This is also a new scientific finding that is very useful for programmers, users and researchers. In general, the main contributions of this research are: 1) proposing an accurate RS using personality traits, demographic characteristics and time-based purchasing behavior; 2) proposing an effective and comprehensive RS for a “clothing” online store; 3) improving the RS performance by solving the cold start issue using personality traits and demographic characteristics; 4) improving the scalability issue in RS through efficient k-nearest neighbors; 5) Mitigating the sparsity issue by using personality traits and demographic characteristics and also by densifying the user-item matrix and 6) improving the RS accuracy by solving the interest drift issue through developing a time-based user-item matrix.

Witness Box Protocol: Automatic machine identification and authentication in industry 4.0

The current wave of Industrial Internet of Things (IIoT) is reshaping the manufacturing sector with system interoperability, remote real-time process monitoring and advanced analytics. As digitally enabled manufacturing machines continue to grow exponentially, it becomes imperative to uniquely and securely identify them in the cyber-physical world, particularly in defense, biomedical, energy and aerospace manufacturing. Research about threats originating from internal adversaries’ i.e the machine/organization owner within a tiered digitally connected supply chain is scarce. This paper introduces a machine fingerprinting scheme named as the ‘Witness Box Protocol’ (WBP) that exploits the physical properties of manufacturing machines (legacy or smart) and their surroundings to create a unique biometric like fingerprint. WBP provides both machine registration and authentication on a digital network through a low cost, non-invasive approach. The fingerprint is generated by a Locality Sensitive Hashing (LSH) technique that accommodates small variations in physical signature data and can corroborate data provenance from machines by verifying machine identity through authentication. Additionally, this fingerprint hash simplifies asset management within a large enterprise or distributed network comprising of thousands of machines in a Manufacturing-as-a-Service (MaaS) paradigm. In this research, fingerprints were randomly generated from the statistical features of signals from 3D printers and CNC machine in a production-like lab environment. Using k-means clustering and Jaccard similarity index, these fingerprints are shown to identify the source equipment with 95 % accuracy.

Privacy-Aware Data Fusion and Prediction With Spatial-Temporal Context for Smart City Industrial Environment

As one of the cyber–physical–social systems that plays a key role in people's daily activities, a smart city is producing a considerable amount of industrial data associated with transportation, healthcare, business, social activities, and so on. Effectively and efficiently fusing and mining such data from multiple sources can contribute much to the development and improvements of various smart city applications. However, the industrial data collected from the smart city are often sensitive and contain partial user privacy such as spatial–temporal context information. Therefore, it is becoming a necessity to secure user privacy hidden in the smart city data before these data are integrated together for further mining, analyses, and prediction. However, due to the inherent tradeoff between data privacy and data availability, it is often a challenging task to protect users’ context privacy while guaranteeing accurate data analysis and prediction results after data fusion. Considering this challenge, a novel privacy-aware data fusion and prediction approach for the smart city industrial environment is put forward in this article, which is based on the classic locality-sensitive hashing technique. At last, our proposal is evaluated by a set of experiments based on a real-world dataset. Experimental results show better prediction performances of our approach compared to other competitive ones.

Diversified service recommendation with high accuracy and efficiency

Collaborative filtering-based recommender systems are regarded as an important tool to predict the items that users will appreciate based on the historical usage of users. However, traditional recommendation solutions often pay more attentions to the accuracy of the recommended items while neglect the diversity of the final recommended list, which may produce partial redundant items in the recommended list and as a result, decrease the satisfaction degree of users. Moreover, historical usage data for recommendation decision-makingoften update frequently, which may lead to low recommendation efficiency as well as scalability especially in the big data environment. Considering these drawbacks, a novel method called DivRec_LSH is proposed in this paper to achieve diversified and efficient recommendations, which is based on the historical usage records and the Locality-Sensitive Hashing (LSH) technique. Finally, we compare our method with existing methods on the MovieLens dataset. Experiment results indicate that our proposal is feasible in addressing the triple dilemmas of recommender systems simultaneously, i.e., high efficiency, accuracy and diversity.

A Fast and Memory-Efficient Spectral Library Search Algorithm Using Locality-Sensitive Hashing.

With the accumulation of MS/MS spectra collected in spectral libraries, the spectral library searching approach emerges as an important approach for peptide identification in proteomics, complementary to the commonly used protein database searching approach, in particular for the proteomic analyses of well-studied model organisms, such as human. Existing spectral library searching algorithms compare a query MS/MS spectrum with each spectrum in the library with matched precursor mass and charge state, which may become computationally intensive with the rapidly growing library size. Here, the software msSLASH, which implements a fast spectral library searching algorithm based on the Locality-Sensitive Hashing (LSH) technique, is presented. The algorithm first converts the library and query spectra into bit-strings using LSH functions, and then computes the similarity between the spectra with highly similar bit-string. Using the spectral library searching of large real-world MS/MS spectra datasets, it is demonstrated that the algorithm significantly reduced the number of spectral comparisons, and as a result, achieved 2-9X speedup in comparison with existing spectral library searching algorithm SpectraST. The spectral searching algorithm is implemented in C/C++, and is ready to be used in proteomic data analyses.

Multi-dimensional quality-driven service recommendation with privacy-preservation in mobile edge environment

With the advance of mobile edge computing (MEC), the number of edge services running on mobile devices grows explosively. In this situation, it is becoming a necessity to recommend the most suitable edge services to a mobile user from massive candidates, based on the historical quality of service (QoS) data. However, historical QoS is a kind of private data for users, which needs to be protected from privacy disclosure. Currently, researchers often use the Locality-Sensitive Hashing (LSH) technique to achieve the goal of privacy-aware recommendations. However, existing LSH-based methods are only applied to the recommendation scenarios with a single QoS dimension (e.g., response time or throughput), without considering the multi-dimensional QoS (e.g., response time and throughput) ensemble, which narrow the application scope of LSH in privacy-preserving recommendations significantly. Considering this drawback, this paper proposes a multi-dimensional quality ensemble-driven recommendation approach named RecLSH-TOPSIS based on LSH and TOPSIS (Technique for Order Preference by Similarity to Ideal Solution) techniques. First, the traditional single-dimensional LSH recommendation approach is extended to be a multi-dimensional one, through which we can obtain a set of candidate services that a user may prefer. Second, we use TOPSIS technique to rank the derived multiple candidate services and return the user an optimal one. At last, a case study is presented to illustrate the feasibility of our proposal to make privacy-preserving edge service recommendations with multiple QoS dimensions.

Privacy-Aware Cross-Platform Service Recommendation Based on Enhanced Locality-Sensitive Hashing

Recommender systems are a promising way for users to quickly find the valuable information that they are interested in from massive data. Concretely, by capturing the user's personalized preferences, a recommender system can return a list of recommended items that best match the user preferences by using collaborative filtering. However, in the big data environment, the heavily fragmented distribution of the QoS (Quality of Services) data for recommendation decision- making presents a large challenge when integrating the QoS data from different platforms while ensuring that the sensitive user information contained in the QoS data is secure. Furthermore, due to the common tradeoff between data availability and privacy in data-driven applications, protecting the sensitive user information contained in the QoS data will probably decrease the availability of QoS data and finally produce inaccurate recommendation results. Considering these challenges, we enhance the classic Locality-Sensitive Hashing (LSH) technique, after which we propose an approach based on enhanced LSH for accurate and less-sensitive cross-platform recommendation decision-makings. Finally, extensive experiments are designed and tested on the reputable WS-DREAM dataset. The test reports prove the benefits of our work compared to other competitive approaches in the aspects of recommendation accuracy, efficiency and privacy protection performances.

RSATree: Distribution-Aware Data Representation of Large-Scale Tabular Datasets for Flexible Visual Query.

Analysts commonly investigate the data distributions derived from statistical aggregations of data that are represented by charts, such as histograms and binned scatterplots, to visualize and analyze a large-scale dataset. Aggregate queries are implicitly executed through such a process. Datasets are constantly extremely large; thus, the response time should be accelerated by calculating predefined data cubes. However, the queries are limited to the predefined binning schema of preprocessed data cubes. Such limitation hinders analysts' flexible adjustment of visual specifications to investigate the implicit patterns in the data effectively. Particularly, RSATree enables arbitrary queries and flexible binning strategies by leveraging three schemes, namely, an R-tree-based space partitioning scheme to catch the data distribution, a locality-sensitive hashing technique to achieve locality-preserving random access to data items, and a summed area table scheme to support interactive query of aggregated values with a linear computational complexity. This study presents and implements a web-based visual query system that supports visual specification, query, and exploration of large-scale tabular data with user-adjustable granularities. We demonstrate the efficiency and utility of our approach by performing various experiments on real-world datasets and analyzing time and space complexity.

Efficient Aggregation Processing in the Presence of Duplicately Detected Objects in WSNs

Wireless sensor networks (WSNs) have received increasing attention in the past decades. Owing to an enhancement of MEMS technology, various types of sensors such as motion detectors, infrared radiation detectors, ultrasonic sensors (sonar), and magnetometers can detect the objects within a certain range. Under such an environment, an object without an identifier can be detected by several sensor nodes. However, existing studies for query processing in WSNs simply assume that the sensing regions of sensors are disjoint. Thus, for query aggregation processing, effective deduplication is vital. In this paper, we propose an approximate but effective aggregate query processing algorithm, called DE-Duplication on the Least Common Ancestor∗ (abbreviated as DELCA∗). In contrast to most existing studies, since we assume that each object does not have a unique identifier, we perform deduplication based on similarity. To recognize the duplicately detected events earlier, we utilize the locality-sensitive hashing (LSH) technique. In addition, since the similarity measures are not generally transitive, we adapt three duplicate semantics. In our experiments, by using a transmission cost model, we demonstrate that our proposed technique is energy-efficient. We also show the accuracy of our proposed technique.

LSH-based private data protection for service quality with big range in distributed educational service recommendations

Service recommendation has become a promising way to extract useful or valuable information from big educational data collected by various sensors and distributed in different platforms. How to protect the private user data in each cluster during recommendation processes is an interesting but challenging problem in the educational domain. A locality-sensitive hashing (LSH) technique has recently been employed to achieve the abovementioned privacy-preservation goal. However, traditional LSH-based recommendation approaches often suffer from low accuracy when the service quality data recruited in recommendations vary in a big range. Considering this drawback, we modify the traditional LSH technique in this paper to make it applicable to the service quality data with a big range, so as to pursue privacy-preserving and an accurate recommended list. Finally, a wide range of experiments are conducted based on the distributed dataset, i.e., WS-DREAM. Experiment results show that our approach can protect the private data in education (e.g., student information in universities) and performs better than other state-of-the-art ones in terms of accuracy and efficiency.