Processing Large Datasets Research Articles

Effect of Architecture and Inference Parameters of Artificial Neural Network Models in the Detection Task on Energy Demand

Artificial neural network models for the task of detection are used in many fields and find various applications. Models of this kind require adequate computational resources and thus require adequate energy expenditure. The increase in the number of parameters, the complexity of architectures, and the need to process large data sets significantly increase energy consumption, which is becoming a key sustainability challenge. Optimization of computing and the development of energy-efficient hardware technologies are essential to reduce the energy footprint of these models. This article examines the effect of the type of model, as well as its parameters, on energy consumption during inference. For this purpose, sensors built into the graphics card were used, and software was developed to measure the energy demand of the graphics card for different architectures of YOLO models (v8, v9, v10), as well as for different batch and model sizes. This study showed that the increase in energy demand is not linearly dependent on batch size. After a certain level of batch size, the energy demand begins to decrease. This dependence does not occur only for n/t size models. Optimum utilization of computing power due to the number of processed images for the studied models occurs at the maximum studied batch size. In addition, tests were conducted on an embedded device.

Edge-Cloud Remote Sensing Data-Based Plant Disease Detection Using Deep Neural Networks with Transfer Learning.

The identification and control of plant diseases have become increasingly complex as agricultural lands expand globally. Traditional manual methods of monitoring and diagnosing diseases are not feasible for large-scale farms. The complexity is further heightened by the heterogeneity of data types collected through remote sensing methods, such as images, videos, and sensor readings, which need to be processed in real-time. This paper introduces a novel EdgeCloud Remote Sensing architecture that utilizes Deep Neural Networks (DNNs) with Transfer Learning to enhance the detection of plant diseases using remote sensing data. The proposed system employs a Fuzzy Deep Convolutional Neural Network (FCDCNN) to process multimodal data collected from both edge nodes and satellite point clouds. Transfer learning allows the sharing of model weights between cloud and edge nodes, thus optimizing performance without requiring frequent retraining. Simulations show that the proposed system achieves a 98% detection accuracy and reduces processing time by 25% compared to existing methods. By distributing tasks between edge and cloud resources, the system is able to process large datasets effectively and improve disease detection performance on vast farmlands.

Optimizing neural network models for predicting nuclear reactor channel temperature: A study on hyperparameter tuning and performance analysis

This study emphasizes how important accurate prediction of channel temperatures in nuclear reactors is for safety and operational efficiency. While traditional methods require long and complex processes such as kernel modeling and mathematical simulations, artificial neural networks (ANN) provide more efficient predictions by accelerating this process. The superior ability of ANNs to process large data sets is intended to demonstrate that this study will provide a valuable alternative compared to conventional methods and increase the accuracy of reactor temperature predictions. In this study, the training performances of Artificial Neural Network (ANN) developed to determine the nuclear reactor channel temperature with different hyperparameter combinations were analysed. It was conducted several experimental studies to assess the influence of hyperparameters on our model for nuclear reactor parameter data prediction. The training and validation results indicates that learning rate, hidden layer sizes and number have critical effects for the more precisive prediction. It was observed that models with a learning rate of 0.05 and 0.5 achieved successful learning with less fluctuation in training and validation errors. When looking at hidden layer sizes, networks with 32 and 64 neurons performed better than networks with 16 neurons. For the test phase our model can successfully predict data with slight error margin. As a result, we demonstrated that neural networks are a powerful tool in nuclear reactor channel temperature prediction through our proposed model.

Machine-learning to analyze human-building interactions: How do people use mobile solar protections?

Buildings significantly impact urban energy consumption. Mobile passive solutions, such as manual solar protections, can mitigate heat gains, but their effectiveness depends on occupants’ decisions. Analyzing occupant adjustments of manual systems is challenging due to the complexity of human behavior. While multiple studies have observed human-building interactions, analyzing large buildings over extended periods remains a technical challenge. This study examines 359 manually controlled solar protections over a year using systematic photographic data. To manage the large volume of images, an unsupervised machine learning algorithm was employed to cluster similar solar protection positions for each window, which were then manually tagged to identify their positions. Results show that solar protections are mostly closed year-round (28 % aperture on average), with minimal differences between occupied and non-occupied days and between warm and cool seasons. Notably, one-third of the solar protections were not operated throughout the year. Our results provide insights into the usage of manual façade systems and offer valuable knowledge for energy simulation. Methodologically, the use of machine-learning algorithms presents a new way to process large datasets of images, emphasizing the importance of prioritizing quality over quantity and strategic data collection. Future research can apply this methodology to different building types and climates to gain broader insights into occupant behavior and solar protection interactions.

DONGLE 2.0: Direct FPGA-Orchestrated NVMe Storage for HLS

Rapid growth in data size poses significant computational and memory challenges to data processing. FPGA accelerators and near-storage processing have emerged as compelling solutions for tackling the growing computational and memory requirements. Many FPGA-based accelerators have shown to be effective in processing large data sets by leveraging the storage capability of either host-attached or FPGA-attached storage devices. However, the current HLS development environment does not allow direct access to host-or FPGA-attached NVMe storage from the HLS code. As such, users must frequently hand off between HLS and host code to access data in storage, and such a process requires tedious programming to ensure functional correctness. Moreover, since the HLS code uses radically different methods to access storage compared to DRAM, the HLS codebase targeting DRAM-based platforms cannot be easily ported to NVMe-based platforms, resulting in limited code portability and reusability. Furthermore, frequent suspension of HLS kernel and synchronization between CPU and FPGA introduce significant latency overhead and require sophisticated scheduling mechanisms to hide latency. To address these challenges, we propose a new HLS storage interface named DONGLE 2.0 that enables direct FPGA-orchestrated NVMe storage access. By providing a unified interface for storage and memory access, DONGLE 2.0 allows a single-source HLS program to target multiple memory/storage devices, thus making the codebase cleaner, portable, and more efficient. DONGLE 2.0 is an extension to DONGLE 1.0 [ 1 ] but adds support for host-attached storage. While its primary focus is still on FPGA NVMe access in near-storage configurations, the added host storage support ensures its compatibility with platforms that lack native support for FPGA-attached NVMe storage. We implemented a prototype of DONGLE 2.0 using an AMD/Xilinx Alveo U200 FPGA and Solidigm DC-P4610 SSD. Our evaluation on various workloads showed a geometric mean speed-up of 2.3× and a reduction in lines of code (LoC) by 2.4× compared to the state-of-the-art commercial platform when using FPGA-attached NVMe storage. Moreover, DONGLE 2.0 demonstrated a geometric mean speed-up of 1.5× and a reduction in LoC by 2.4× compared to the state-of-the-art commercial platform when using host-attached NVMe storage.

Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms

The paper "Quantum Machine Learning: Leveraging Quantum Computing for Enhanced Learning Algorithms" explores the integration of quantum computing principles into classical machine learning techniques, aiming to address limitations such as scalability and computational inefficiency. It presents the foundational concepts of quantum computing, including superposition and entanglement, and their application in accelerating machine learning processes. The study emphasizes the potential for quantum algorithms to significantly improve the performance of machine learning tasks by processing large datasets more efficiently and exploring larger hypothesis spaces. Key quantum machine learning algorithms discussed include Quantum Support Vector Machines (QSVM), Quantum Principal Component Analysis (QPCA), and Quantum Neural Networks (QNN), each of which leverages quantum mechanics to overcome the computational barriers faced by classical algorithms. The Quantum Approximate Optimization Algorithm (QAOA) is also highlighted for its ability to optimize machine learning models more effectively. While the theoretical benefits of Quantum Machine Learning (QML) are promising, the practical application of these techniques is currently limited by the constraints of existing quantum hardware. This research contributes to the emerging field of QML by examining its potential advantages and future implications in addressing complex data processing challenges.

Enhanced Early Detection of Thyroid Abnormalities using a Hybrid Deep Learning Model: A Sequential CNN and K-Means Clustering Approach

The thyroid gland, often referred to as the butterfly gland due to its shape, is located in the neck and plays a crucial role in regulating metabolism. It is susceptible to various health conditions, including hypothyroidism, hyperthyroidism, thyroid cancer, and thyroid nodules. Early detection of these conditions is essential for accurate diagnosis and effective treatment. Detecting thyroid nodules using machine learning and deep learning techniques presents a challenging yet promising research avenue. The choice of model depends on the characteristics of the patient's thyroid data, the dataset size, and the available computational resources. Hybrid models can be employed to handle complex data more effectively. In this study, a sequential Convolutional Neural Network (CNN) model was developed due to its capability to automate feature extraction and focus on Regions-of-Interest (ROIs) for detecting thyroid abnormalities. The proposed model achieved an accuracy of 81.5%, with a precision of 97.4% and a sensitivity of 83.1%, indicating its robustness in classifying images as benign or malignant. The confusion matrix provided further performance insights. Data segmentation was enhanced using K-means clustering for its scalability and efficiency in processing large medical image datasets. Compared to traditional models, the proposed hybrid approach demonstrated a significant improvement in diagnostic accuracy and precision, achieving performance gains of approximately 15-20% over baseline methods. These advancements underscore the potential of integrating machine learning and deep learning in medical diagnostics, paving the way for more reliable and efficient diagnostic tools for healthcare professionals.

Application of transformers in stomatological imaging: A review

Stomatology extensively utilizes detailed imaging to assist clinicians, traditionally requiring labor-intensive manual analysis, which significantly adds to their workload. Transformers are revolutionary neural network in deep learning, gaining substantial momentum in computer vision tasks. Their introduction into medical imaging, particularly in processing large image datasets, has been marked by remarkable precision and efficiency, establishing them as a pivotal tool in emerging research. However, the application of Transformers in stomatological imaging is still in its infancy. Current studies primarily focus on segmenting specific anatomical features such as teeth and jawbones, with some clinical implementations. Yet, the comprehensive analytical potential of Transformers in this field remains largely untapped. This paper presents an introductory examination of Transformers, coupled with an initial synthesis and assessment of its dental applications across various areas. It will highlight the observed advantages and limitations in dental contexts and conclude with a discussion on future research directions. This serves as a foundational guide for in-depth future investigations in this area.

PRIVACY-PRESERVING TECHNIQUES IN BIG DATA AND INFORMATION SECURITY

In the era of big data, ensuring privacy while processing vast amounts of sensitive information poses a significant challenge. Traditional encryption methods often fall short in maintaining both privacy and data utility during computation. This paper introduces Two-Trapdoor Homomorphic Encryption (TTHE), a novel approach designed to enhance privacy-preserving capabilities in big data and information security. TTHE combines the strengths of trapdoor functions with homomorphic encryption to enable secure data processing without compromising privacy. With the exponential growth of data, safeguarding sensitive information has become a critical concern. Existing encryption schemes often struggle to balance between privacy preservation and computational efficiency. Homomorphic encryption offers a potential solution by allowing computations on encrypted data, but current methods are limited by performance and scalability issues. The key challenge addressed is the inefficiency and performance bottlenecks in current homomorphic encryption schemes, which hinder their practical application in big data environments. Traditional methods often face limitations in processing large datasets efficiently while maintaining robust security. TTHE is proposed as an enhancement over traditional homomorphic encryption. It integrates two distinct trapdoor functions to provide a dual-layer security approach, enabling efficient and scalable computation on encrypted data. The method involves a novel encryption scheme where operations on ciphertexts are performed without decryption, preserving data privacy throughout the process. Extensive experiments demonstrate that TTHE significantly improves both computational efficiency and security. The proposed method achieved a processing speed increase of 45% compared to conventional homomorphic encryption schemes. Additionally, TTHE maintained a privacy level with a security strength of 128-bit encryption, providing robust protection against potential attacks.

Large Language Models Can Enable Inductive Thematic Analysis of a Social Media Corpus in a Single Prompt: Human Validation Study.

Manually analyzing public health-related content from social media provides valuable insights into the beliefs, attitudes, and behaviors of individuals, shedding light on trends and patterns that can inform public understanding, policy decisions, targeted interventions, and communication strategies. Unfortunately, the time and effort needed from well-trained human subject matter experts makes extensive manual social media listening unfeasible. Generative large language models (LLMs) can potentially summarize and interpret large amounts of text, but it is unclear to what extent LLMs can glean subtle health-related meanings in large sets of social media posts and reasonably report health-related themes. We aimed to assess the feasibility of using LLMs for topic model selection or inductive thematic analysis of large contents of social media posts by attempting to answer the following question: Can LLMs conduct topic model selection and inductive thematic analysis as effectively as humans did in a prior manual study, or at least reasonably, as judged by subject matter experts? We asked the same research question and used the same set of social media content for both the LLM selection of relevant topics and the LLM analysis of themes as was conducted manually in a published study about vaccine rhetoric. We used the results from that study as background for this LLM experiment by comparing the results from the prior manual human analyses with the analyses from 3 LLMs: GPT4-32K, Claude-instant-100K, and Claude-2-100K. We also assessed if multiple LLMs had equivalent ability and assessed the consistency of repeated analysis from each LLM. The LLMs generally gave high rankings to the topics chosen previously by humans as most relevant. We reject a null hypothesis (P<.001, overall comparison) and conclude that these LLMs are more likely to include the human-rated top 5 content areas in their top rankings than would occur by chance. Regarding theme identification, LLMs identified several themes similar to those identified by humans, with very low hallucination rates. Variability occurred between LLMs and between test runs of an individual LLM. Despite not consistently matching the human-generated themes, subject matter experts found themes generated by the LLMs were still reasonable and relevant. LLMs can effectively and efficiently process large social media-based health-related data sets. LLMs can extract themes from such data that human subject matter experts deem reasonable. However, we were unable to show that the LLMs we tested can replicate the depth of analysis from human subject matter experts by consistently extracting the same themes from the same data. There is vast potential, once better validated, for automated LLM-based real-time social listening for common and rare health conditions, informing public health understanding of the public's interests and concerns and determining the public's ideas to address them.

Computer-assisted regional-residual gravity anomaly separation with regularized first- and second-order derivatives

The regional-residual separation of gravity anomalies in crustal and mineral exploration was a graphical-based procedure before the advent of fast digital computers and the need for more efficient algorithms to process large data sets. However, since requiring the supervision of an experienced interpreter, the results, once obtained with graphical procedures, are often accepted as second to none in producing anomalies with geologic significance. Numerical methods based on spectral filtering and robust polynomial fitting have worked in many scenarios but seem not fully effective in replicating in algorithms the kind of results once obtained with interpreter-assisted graphical methods. We develop a computer-assisted regional residual separation (CARRS) procedure implemented by a set of short MATLAB scripts; in many aspects, CARRS simulates the operations of former graphical methods but requires few decisions and minor handwork from the interpreter. CARRS applies robust polynomial fitting to points with a low horizontal gradient and a vertical second-order derivative; thus, selecting fitting points as a real interpreter would do with graphical approaches in outlining the regional field. A regularized procedure is used to calculate the stable first- and second-order derivatives. Companion MATLAB codes allow for the replication of our results and further exploration with different threshold levels to identify flat domain regions. CARRS is illustrated with airborne gravity data CPRM-1123, freely available to download. A data window of the residual field is used to analyze the distribution of cassiterite deposits in greisen zones of the Paleoproterozoic Velho Guilherme granites in the Amazon Craton, as their distribution appears in the observed gravity anomaly and its corresponding residual field.

A digital twin-based traffic light management system using BIRCH algorithm

Urban transportation networks are vital for the economic and environmental well-being of cities and they are faced with the integration of Human-Driven Vehicles (HVs) and Connected and Autonomous Vehicles (CAVs) challenge. Most of the traditional traffic management systems fail to effectively manage the dynamic and complex flows of mixed traffic, mainly because of large computational requirements and the restrictions that control models of traffic lights directly based on extensive and continuous training data. Most of the times, the operational flexibility of CAVs is severely compromised for the safety of HVs, or CAVs are given high priority without taking into account the efficiency of HVs leading to lower performance, especially at low CAV penetration rates. On the other hand, the existing adaptive traffic light approaches were usually partial and could not adapt to the real-time behaviors of the traffic system. Some systems operate with inflexible temporal control plans that cannot react to variations in traffic flow or use adaptive control strategies that are based on a limited set of static traffic conditions. This paper presents a novel traffic light control approach utilizing the BIRCH (Balanced Iterative Reducing and Clustering using Hierarchies) clustering algorithm combined with digital twins for a more adaptive and efficient system. The BIRCH is effective in processing large datasets because it clusters data points incrementally and dynamically into a small set of representatives. The suggested method does not only enable better simulation and prediction of traffic patterns but also makes possible the real-time adaptive control of traffic signals at signalized intersections. It also improves traffic flow, reduces congestion, and minimizes vehicle idling time by adjusting the green and red light durations dynamically based on both real-time and historical traffic data. This approach is assessed under different traffic intensities, which include low, moderate, and high, while efficiency, fuel consumption, and the number of stops are being compared with the traditional and the existing adaptive traffic management systems.

Advancements, Applications, and Future Directions of Artificial Intelligence in Healthcare

Background: The integration of artificial intelligence (AI) into healthcare represents a transformative shift in medical procedures, offering substantial benefits across various domains. With advancements in AI technologies such as machine learning (ML), deep learning (DL), and natural language processing (NLP), healthcare systems are witnessing improvements in early detection, patient treatment, and overall administration. This article traces the evolution of AI, from foundational contributions by Alan Turing during World War II to contemporary applications like ChatGPT, and examines the impact of AI in enhancing diagnostic accuracy and treatment outcomes. Methods: This comprehensive review analyzes the existing literature on AI applications in healthcare, focusing on various AI methodologies and their integration into clinical settings. It evaluates the effectiveness of AI in processing large datasets, improving diagnostic precision, and facilitating data-driven decision-making. The study also explores the ethical, legal, and technical challenges associated with AI deployment in medical environments. Results: AI technologies have demonstrated significant improvements in healthcare, particularly in early disease detection, personalized treatment plans, and resource management. The use of AI in analyzing vast medical datasets has enhanced diagnostic accuracy, reduced costs, and optimized patient care. However, challenges related to ethical considerations, patient privacy, and system reliability remain critical barriers to full-scale AI adoption. Conclusion: Despite the challenges, AI is positioned as an indispensable tool in modern medicine, capable of enhancing preventive care, personalizing treatments, and improving healthcare delivery. This review proposes a framework for evaluating the benefits, challenges, and strategies of AI integration in healthcare. Further research is essential to maximize AI's potential while addressing ethical and practical concerns, ensuring safe and effective implementation in clinical settings.

Simple (yet Efficient) Function Authoring for Vectorized Engines

Vectorized execution engines process large datasets by decomposing computations into concise (tight) loops, which can be more efficiently executed by modern hardware. Providing loops that are optimal for execution usually adds burden to the software development process, as developers are required to understand details of vectorized execution, columnar data layout, data encodings, and the code compilation process itself, presenting a steep learning curve and challenges to organizations building and scaling large engineering teams. Due to their large quantity, scalar function authoring accentuates this problem. In our experience building the Velox open source execution engine, we have observed that exposing a large number of developers to the complexity inherent to vectorization resulted in a disproportionate amount of bugs and performance inefficiencies. In this paper, we describe the simple function interface (SFI) created to address this issue. SFI highly simplifies scalar function authoring by encapsulating the vectorization complexity required to generate tight loops, and presenting developers with a simpler, conciser, and more natural row-based interface - without sacrificing performance. SFI also hides columnar layout details, while providing developers the flexibility to efficiently implement advanced features such as functions with nested and recursive parameter types, type variables, variadic parameters, and generic types. Today, more than a thousand functions have been added to Velox using the SFI, implementing popular open source SQL dialects and internal domain-specific use cases at Meta, and are in active production use. While this paper presents implementation details, performance pitfalls, experimental results, and our overall experience developing the state-of-the-art Velox vectorized execution engine, we believe the concepts and trade-offs to be fundamentally equivalent and generally applicable to other vectorized engines.

Comparing location-specific and location-open social media data: methodological lessons from a study of blaming of minorities on Twitter during the COVID-19 pandemic

Social media platforms such as Twitter (currently X) have become important sites of public discourse and participation. Researchers have attempted to identify and collect Twitter data within a certain country or region in order to answer research questions within a particular locale. However, location information of tweets is limited. Tackling the case of public blaming of minorities on Twitter in the context of the COVID-19 pandemic in the UK, we present a method for identifying UK-based tweets and analyse two types of datasets that we collected and processed: (a) tweets with UK location-tags (labelled as location-specific data and referred to as UK datasets); and (b) tweets with UK location-tags and / or user profiles containing potential UK location information (labelled as location-open data and referred to as ALL datasets). The empirical results reveal that the overall sentiments in the two dataset types align in the same direction, but the location-specific datasets contain more extreme discourses (i.e., more positive and more negative sentiments and fewer neutral sentiments). Furthermore, in the location-specific datasets, the range of theme areas is narrower, although the themes still grasp the essence of the discussion about blaming minorities found in the larger dataset. The findings demonstrate strengths and limitations of the two dataset types and that the location-specific data can be suitable especially when the available research resources are insufficient for collecting or processing larger datasets. Nevertheless, we propose that future research may consider comparing smaller and bigger datasets to test differences between these for other topics for which specific locations may be of particular interest.

A Software Tool For Automated Analysis And Characterization Of Raw PIV Images

The time to process large PIV data sets can be expensive and the resultant velocity fields are a strong function of the quality of the particle images used. For new and even experienced users of PIV, determination of the quality of a particle image data set can be challenging and time consuming especially for a large number of data sets. This is compounded with new high-speed camera systems that are capable of collecting terabytes of data quickly. A software tool is described here that allows the user to make informed decisions on the general quality of the data sets and what pre-processing image data steps are needed. The software provides statistical feedback on such parameters as particle count, particle size, particle intensity for not only a single image but also a complete data set. Using this software allows the user to make informed decisions and generate and document the quality of the data collected. Based on this, a robust PIV processing algorithm can be developed.

Predictive analytics for traffic flow optimization in urban logistics: A transformer-based time series approach.

In this study, we focus on the analysis and prediction of urban logistics traffic flow, a field that is gaining increasing attention due to the acceleration of global urbanization and heightened environmental awareness. Existing forecasting methods face challenges in processing large and complex datasets, particularly when extracting and analyzing valid information from these data, often hindered by noise and outliers. In this context, time series analysis, as a key technique for predicting future trends, becomes crucial for supporting real-time traffic management and long-term traffic planning. To this end, we propose a composite network model that integrates gated recurrent unit (GRU), autoregressive integrated moving average (ARIMA), and temporal fusion transformer (TFT), namely the GRU-ARIMA-TFT network model, to enhance prediction accuracy and efficiency. Through the analysis of experimental results on different datasets, we demonstrate the significant advantages of this model in improving prediction accuracy and understanding complex traffic patterns. This research not only theoretically expands the boundaries of urban logistics traffic flow prediction but also holds substantial practical significance in real-world applications, especially in optimizing urban traffic planning and logistics distribution strategies during peak periods and under complex traffic conditions. Our study provides a robust tool for addressing real-world issues in the urban logistics domain and offers new perspectives and methodologies for future urban traffic management and logistics system planning.

ПРОГНОЗИРОВАНИЕ ВЫРУЧКИ БЫСТРОРАСТУЩЕЙ КОМПАНИИ С ИСПОЛЬЗОВАНИЕМ ЛОГИСТИЧЕСКОЙ КРИВОЙ

The aim of the study is to assess the advancement potential of fast-growing companies, to create prerequisites for predicting the gain of such companies. The objective is to identify and forecast the growth points of the country’s economy, which is generally based on the advancement of individual economic entities, namely organizations. The research method for forecasting revenue as the main indicator of fast-growing companies (FGC) is considered. As an approximation, a logistic curve (the Ferhulst curve) is examined, the parameters of which are identified by the least squares method. Open data from financial statements of organizations are applied as a source. This work uses the criterion for distinguishing fast-growing companies; the average annual growth is of at least 50% at current prices. The novelty of the work lies in studying fast-growing organizations using a logistic curve (sigmoid). Approximation parameters are identified. A growth assessment (forecast) for organizations for 2040 is made. The statistical reliability of the selected approximation is established for 714 large and medium-sized fast-growing companies from the sample. The research results state, that a sample of about 900 fast-growing Russian companies is identified using the methods for processing large data sets from 2.5 million organizations. The illustrative calculations are based on actual sample data, highlighting the example of Wildberries in more detail. An assessment of the growth opportunities of the organizations under consideration is made. Studying the applicability of the proposed method using statistical criteria is carried out. The findings show that studying fast-growing organizations’ advancement for a medium or longer period using a logistic curve looks more preferable than applying an exponential one. For the vast majority of such organizations, the approximation of the logistic curve is statistically significant.

A succinct and approximate greedy algorithm for the Minimum Set Cover Problem

The Minimum Set Cover Problem (MSCP) is a combinatorial optimization problem belonging to the NP-Hard class in computer science. For this reason, there is no algorithm that in the worst case ensures finding an optimal solution in polynomial-time. For a given universe X, the popular greedy heuristic, called Greedy-SetCover, is the main theoretical contribution to obtain an approximate solution for the MSCP in polynomial-time, offering an optimal approximate ratio of (ln|X|+1). In this article, we propose an approximate algorithm for MSCP within a succinct representation of the input dataset, whose empirical performance improves Greedy-SetCover both in quality and execution time, while offering the same optimal approximation ratio for the problem. Our experiments show that the proposed algorithm is magnitudes of times faster than the aforementioned greedy one, obtaining on average a cardinality much closer to the optimal solution. Furthermore, because we work on a succinct representation that allows us to compute operations between sets using bitwise operators, we can process much larger datasets than state-of-the-art solutions. As a result, our proposal is also a suitable alternative for processing large datasets as required by the current Big Data era.

Average nucleotide identity-based Staphylococcus aureus strain grouping allows identification of strain-specific genes in the pangenome.

We analyzed the genomic diversity of Staphylococcus aureus, a globally prevalent bacterial species that causes serious infections in humans. Our goal was to build a genetic picture of the different strains of S. aureus and which genes may be associated with them. We reprocessed >84,000 genomes and subsampled to remove redundancy. We found that individual samples sharing >99.5% of their genome could be grouped into strains. We also showed that a portion of genes that are present in intermediate frequency in the species are strongly associated with some strains but completely absent from others, suggesting a role in strain specificity. This work lays the foundation for understanding individual gene histories of the S. aureus species and also outlines strategies for processing large bacterial genomic data sets.