Usage Of Computing Resources Research Articles

UniSchedApi: A comprehensive solution for university resource scheduling and methodology comparison

This paper introduces UniSchedApi, an API-based solution that revolutionizes optimized university resource scheduling. The primary focus of the research is the detailed evaluation of two automatic resource allocation methods: Tabu Search (TS) and Genetic Algorithm (GA). The paper thoroughly explores how these methods address challenges associated with resource allocation in university environments, considering critical factors such as teacher availability, student time constraints, classroom features (including computers, projectors, TV's, specialized laboratories, specialized equipment, etc.), among others. The evaluation is carried out meticulously, measuring the performance and memory resource usage of both algorithms, considering the comparison with the manual scheduling. The results reveal that the TS algorithm excels in terms of temporal efficiency and computational resource usage. Based on these findings, UniSchedApi implements GA and TS but uses TS as the default algorithm, ensuring more efficient and optimized management of academic resources. This research not only presents a practical solution with UniSchedApi but also provides a deep understanding of the methods for evaluating and selecting algorithms to address specific challenges in university resource allocation. These results lay the groundwork for future improvements in academic resource management.

MiniSNV: accurate and fast single nucleotide variant calling from nanopore sequencing data.

Nanopore sequence technology has demonstrated a longer read length and enabled to potentially address the limitations of short-read sequencing including long-range haplotype phasing and accurate variant calling. However, there is still room for improvement in terms of the performance of single nucleotide variant (SNV) identification and computing resource usage for the state-of-the-art approaches. In this work, we introduce miniSNV, a lightweight SNV calling algorithm that simultaneously achieves high performance and yield. miniSNV utilizes known common variants in populations as variation backgrounds and leverages read pileup, read-based phasing, and consensus generation to identify and genotype SNVs for Oxford Nanopore Technologies (ONT) long reads. Benchmarks on real and simulated ONT data under various error profiles demonstrate that miniSNV has superior sensitivity and comparable accuracy on SNV detection and runs faster with outstanding scalability and lower memory than most state-of-the-art variant callers. miniSNV is available from https://github.com/CuiMiao-HIT/miniSNV.

Proposal of a Cost-Effective and Adaptive Customized Driver Inattention Detection Model Using Time Series Analysis and Computer Vision

Advanced Driver Assistance Systems, such as Forward Collision Warning and Lane Departure Warning, play a crucial role in accident prevention by alerting drivers to potential hazards. With the advent of fully autonomous driving technology that requires no driver input, there is now a greater emphasis on monitoring the state of vehicle occupants. This is particularly important because, in emergency situations where control must suddenly be transferred to an unprepared occupant, the risk of accidents increases significantly. To mitigate this risk, new monitoring technologies are being developed to analyze driver behavior and detect states of inattention or drowsiness. In response to the emerging demands of driver monitoring technology, we have developed the Customized Driver Inattention Detection Model (CDIDM). This model employs video analysis and statistical techniques to accurately and rapidly classify information on drivers’ gazes. The CDIDM framework defines the components of inattentive or drowsy driving based on the Driver Monitoring System (DMS) safety standards set by the European New Car Assessment Programme (EuroNCAP). By defining six driving behavior-related scenarios, we have improved the accuracy of driver inattention assessment. The CDIDM estimates the driver’s gaze while simultaneously analyzing data in real-time. To minimize computational resource usage, this model incorporates a series of preprocessing steps that facilitate efficient time series data analysis, utilizing techniques such as DTW Barycenter Averaging (DBA) and K-means clustering. This results in a robust driver attention monitoring model based on time series classification.

Enhanced you only look once approach for automatic phytoplankton identification

<p>Conventionally, identifying phytoplankton species is challenging due to human taxonomical knowledge limitations. Advanced technology can overcome this problem. A novel model that accurately enhances phytoplankton detection and identification classification by combining asymmetric convolution and vision transformers (ACVIT) within the YOLOv8m framework is promoted with ACVIT-YOLO. The performance of this model surpasses the original YOLOv8m model, exhibiting a notable 2.4% enhancement in precision, 5.5% improvement in recall, and 1.1% gain in mAP 50 score. The enhanced effectiveness of ACVIT-YOLO compared to the YOLOv8m model, further demonstrated by the decreased giga floating-point operations (GFLOP), decreased parameter count, and compact dimensions, significantly improves the automation of phytoplankton species identification. This suggests that the ACVIT-YOLO model could produce a better prediction system for identifying phytoplankton with similar accuracy to the original YOLOv8m model but with lower computational power and resource usage.</p>

Softwarized and containerized microservices-based network management analysis with MSN

Microservice architecture is a service-oriented paradigm that enables the decomposition of cumbersome monolithic-based software systems. Using microservice design principles, it is possible to develop flexible, scalable, reusable, and loosely coupled software that could be containerized and deployed in a distributed edge/cloud environment. The flexible deployment of microservices in an edge environment increases system performance in terms due to dynamic service function placement and chaining possibly resulting in latency reduction, fault tolerance, scalability, efficient resource utilization, cost reduction, and energy consumption reduction. On the other hand, virtualization and containerization of microservices add processing and communication overheads. Therefore, to evaluate end-to-end microservices-based system performance, we need to have an end-to-end mathematical formulation of the overall microservice-based network system. Incorporating the virtualization overhead, here we provide end-to-end mathematical formulation considering system parameters: latency, throughput, computational resource usage, and energy consumption. We then evaluate the formulation in a testbed environment with the Microservice-based SDN (MSN) framework that decomposes the Software-defined Networking (SDN) controller in microservices with Docker Container. The final result validates the presented mathematical modeling of the system’s dynamic behavior which can be used to design a microservice-based system.

Beyond Docker: Enhancing vantage6 with Kubernetes for Federated Learning.

Vantage6, a powerful platform for privacy-preserving analysis in the life-sciences domain, employs Docker in its data nodes when performing tasks like federated learning. This tight bond to Docker poses challenges on infrastructure security, efficient computing resources usage, and integration of alternative container technologies. To overcome these challenges, we explored the integration of Kubernetes into vantage6 through a Proof-of-Concept (PoC) approach. The PoC designed and implemented a Kubernetes-based architecture for vantage6, which can be easily deployed on a single machine or a cluster. This PoC serves as the reference to accelerate the adoption of Kubernetes in vantage6.

Enhanced Dynamic Programming Approaches for Efficient Solutions to the Traveling Salesman Problem

This study aims to enhance dynamic programming techniques for efficiently solving the Traveling Salesman Problem, a fundamental combinatorial optimization challenge. Given its NP-hard classification, traditional exact algorithms become computationally infeasible as the problem size increases. The research revisits foundational dynamic programming principles, notably the Held-Karp algorithm, and identifies existing limitations. The study begins with a comprehensive literature review, followed by an analysis of the dynamic programming challenges specific to TSP. Novel algorithms are then developed, implemented, and rigorously tested against benchmark instances. Performance evaluation is conducted using metrics such as execution time, memory usage, and solution optimality across different problem sizes. Results demonstrate significant improvements in efficiency and scalability, with enhanced algorithms achieving optimal solutions in reduced time and computational resource usage. However, the exponential growth in complexity remains a challenge for larger instances. The study concludes with recommendations for future research, focusing on further algorithmic refinements and exploring hybrid approaches to address large-scale TSPs.

SLIVER: Unveiling large scale gene regulatory networks of single-cell transcriptomic data through causal structure learning and modules aggregation

Prevalent Gene Regulatory Network (GRN) construction methods rely on generalized correlation analysis. However, in biological systems, regulation is essentially a causal relationship that cannot be adequately captured solely through correlation. Therefore, it is more reasonable to infer GRNs from a causal perspective. Existing causal discovery algorithms typically rely on Directed Acyclic Graphs (DAGs) to model causal relationships, but it often requires traversing the entire network, which result in computational demands skyrocketing as the number of nodes grows and make causal discovery algorithms only suitable for small networks with one or two hundred nodes or fewer. In this study, we propose the SLIVER (cauSaL dIscovery Via dimEnsionality Reduction) algorithm which integrates causal structural equation model and graph decomposition. SLIVER introduces a set of factor nodes, serving as abstractions of different functional modules to integrate the regulatory relationships between genes based on their respective functions or pathways, thus reducing the GRN to the product of two low-dimensional matrices. Subsequently, we employ the structural causal model (SCM) to learn the GRN within the gene node space, enforce the DAG constraint in the low-dimensional space, and guide each factor to aggregate various functions through cosine similarity. We evaluate the performance of the SLIVER algorithm on 12 real single cell transcriptomic datasets, and demonstrate it outperforms other 12 widely used methods both in GRN inference performance and computational resource usage. The analysis of the gene information integrated by factor nodes also demonstrate the biological explanation of factor nodes in GRNs. We apply it to scRNA-seq of Type 2 diabetes mellitus to capture the transcriptional regulatory structural changes of β cells under high insulin demand.

Evaluating text classification: A benchmark study

This paper presents an impartial and extensive benchmark for text classification involving five different text classification tasks, 20 datasets, 11 different model architectures, and 42,800 algorithm runs. The five text classification tasks are fake news classification, topic detection, emotion detection, polarity detection, and sarcasm detection. While in practice, especially in Natural Language Processing (NLP), research tends to focus on the most sophisticated models, we hypothesize that this is not always necessary. Therefore, our main objective is to investigate whether the largest state-of-the-art (SOTA) models are always preferred, or in what cases simple methods can compete with complex models, i.e. for which dataset specifications and classification tasks. We assess the performance of different methods with varying complexity, ranging from simple statistical and machine learning methods to pretrained transformers like robustly optimized BERT (Bidirectional Encoder Representations from Transformers) pretraining approach (RoBERTa). This comprehensive benchmark is lacking in existing literature, with research mainly comparing similar types of methods. Furthermore, with increasing awareness of the ecological impacts of extensive computational resource usage, this comparison is both critical and timely.We find that overall, bidirectional long short-term memory (LSTM) networks are ranked as the best-performing method albeit not statistically significantly better than logistic regression and RoBERTa. Overall, we cannot conclude that simple methods perform worse although this depends mainly on the classification task. Concretely, we find that for fake news classification and topic detection, simple techniques are the best-ranked models and consequently, it is not necessary to train complicated neural network architectures for these classification tasks. Moreover, we also find a negative correlation between F1 performance and complexity for the smallest datasets (with dataset size less than 10,000). Finally, the different models’ results are analyzed in depth to explain the model decisions, which is an increasing requirement in the field of text classification.

Comparative Study of Keras CNNs for Tuberculosis Detection from Chest X-rays

As a major global health burden, tuberculosis (TB) requires prompt and accurate diagnostic solutions. The effectiveness of Convolutional Neural Networks (CNNs) for TB prediction from chest X-ray images is investigated in this work using a variety of Keras applications. We compare the accuracy, efficiency, and computational resource usage of the VGG16, ResNet50, and EfficientNetV2B2 architectures. By means of comprehensive testing and analysis on a wide range of datasets, we determine the applicability of every model for tuberculosis identification. The findings show that EfficientNetV2B2 is the most promising architecture, achieving remarkable accuracy (99.5%) with enhanced computational efficiency, while VGG16 and ResNet50 show competitive performance. These results highlight how deep learning-based methods have the potential to transform tuberculosis diagnosis by providing doctors with a trustworthy and usable instrument for early detection and treatment. Our research has implications for improving healthcare outcomes and bolstering international efforts to fight tuberculosis. Keywords—EfficientNet, CNN, VGG16,Keras, Accuracy

Multi-Task Scenario Encrypted Traffic Classification and Parameter Analysis.

The widespread use of encrypted traffic poses challenges to network management and network security. Traditional machine learning-based methods for encrypted traffic classification no longer meet the demands of management and security. The application of deep learning technology in encrypted traffic classification significantly improves the accuracy of models. This study focuses primarily on encrypted traffic classification in the fields of network analysis and network security. To address the shortcomings of existing deep learning-based encrypted traffic classification methods in terms of computational memory consumption and interpretability, we introduce a Parameter-Efficient Fine-Tuning method for efficiently tuning the parameters of an encrypted traffic classification model. Experimentation is conducted on various classification scenarios, including Tor traffic service classification and malicious traffic classification, using multiple public datasets. Fair comparisons are made with state-of-the-art deep learning model architectures. The results indicate that the proposed method significantly reduces the scale of fine-tuning parameters and computational resource usage while achieving performance comparable to that of the existing best models. Furthermore, we interpret the learning mechanism of encrypted traffic representation in the pre-training model by analyzing the parameters and structure of the model. This comparison validates the hypothesis that the model exhibits hierarchical structure, clear organization, and distinct features.

BSEO-Semiautomatic Method for Determination of Oil Recovery with Nanofluids in Microfluidic Devices.

Microfluidic models have become essential instruments for studying enhanced oil recovery techniques through fluid and chemical injection into micromodels to observe interactions with pore structures and resident fluids. The widespread use of cost-effective lab-on-a-chip devices, known for efficient data extraction and minimal reagent usage, has driven demand for efficient data management methods crucial for high-performance data and image analyses. This article introduces a semiautomatic method for calculating oil recovery in polymeric nanofluid flooding experiments based on the background subtraction (BSEO). It employs the background subtraction technique, generating a foreground binary mask to detect injected fluids represented as pixel areas. The pixel difference is then compared to a threshold value to determine whether the given pixel is foreground or background. Moreover, the proposed method compares its performance with two other representative methods: the ground truth (manual segmentation) and Fiji-ImageJ software. The experiments yielded promising results. Low values of mean-squared error (MSE), mean absolute error (MAE), and root-mean-squared error (RMSE) indicate minimal prediction errors, while a substantial coefficient of determination (R2) of 98% highlights the strong correlation between the method's predictions and the observed outcomes. In conclusion, the presented method emphasizes the viability of BSEO as a robust alternative, offering the advantages of reduced computational resource usage and faster processing times.

Deep Reinforcement Learning-Empowered Cost-Effective Federated Video Surveillance Management Framework.

Video surveillance systems are integral to bolstering safety and security across multiple settings. With the advent of deep learning (DL), a specialization within machine learning (ML), these systems have been significantly augmented to facilitate DL-based video surveillance services with notable precision. Nevertheless, DL-based video surveillance services, which necessitate the tracking of object movement and motion tracking (e.g., to identify unusual object behaviors), can demand a significant portion of computational and memory resources. This includes utilizing GPU computing power for model inference and allocating GPU memory for model loading. To tackle the computational demands inherent in DL-based video surveillance, this study introduces a novel video surveillance management system designed to optimize operational efficiency. At its core, the system is built on a two-tiered edge computing architecture (i.e., client and server through socket transmission). In this architecture, the primary edge (i.e., client side) handles the initial processing tasks, such as object detection, and is connected via a Universal Serial Bus (USB) cable to the Closed-Circuit Television (CCTV) camera, directly at the source of the video feed. This immediate processing reduces the latency of data transfer by detecting objects in real time. Meanwhile, the secondary edge (i.e., server side) plays a vital role by hosting a dynamically controlling threshold module targeted at releasing DL-based models, reducing needless GPU usage. This module is a novel addition that dynamically adjusts the threshold time value required to release DL models. By dynamically optimizing this threshold, the system can effectively manage GPU usage, ensuring resources are allocated efficiently. Moreover, we utilize federated learning (FL) to streamline the training of a Long Short-Term Memory (LSTM) network for predicting imminent object appearances by amalgamating data from diverse camera sources while ensuring data privacy and optimized resource allocation. Furthermore, in contrast to the static threshold values or moving average techniques used in previous approaches for the controlling threshold module, we employ a Deep Q-Network (DQN) methodology to manage threshold values dynamically. This approach efficiently balances the trade-off between GPU memory conservation and the reloading latency of the DL model, which is enabled by incorporating LSTM-derived predictions as inputs to determine the optimal timing for releasing the DL model. The results highlight the potential of our approach to significantly improve the efficiency and effective usage of computational resources in video surveillance systems, opening the door to enhanced security in various domains.

Asynchronous Federated Learning: Methods and applications

Federated Learning (FL) is a distributed alternative to traditional machine learning frameworks that computes a global model on a centralized aggregation server according to the parameters of local models, which can address the privacy leakage problem caused by collecting sensitive data from local devices. However, the classic FL methods with synchronous aggregation strategies, in many cases, shall suffer from limitations in resource utilization due to the need to wait for slower devices (stragglers) to aggregate during each training epoch. In addition, the accuracy of the global model can be affected by the uneven distribution of data among unreliable devices in real-world scenarios. Therefore, many Asynchronous Federated Learning (AFL) methods have been developed on many occasions to improve communication efficiency, model performance, privacy, and security. This article elaborates on the existing research on AFL and its applications in many areas. The paper first introduces the concept and development of FL, and then discusses in detail the related work and main research directions of AFL, including dealing with stragglers, staleness, communication efficiency between devices, and privacy and scalability issues. Then, this paper also explores the application of AFL in different fields, especially in the fields of mobile device edge computing, Internet of Things devices, and medical data analysis. Finally, the article gives some outlook on future research directions and believes that it is necessary to design efficient asynchronous optimization algorithms, reduce communication overhead and computing resource usage, and explore new data privacy protection methods.

Exploiting Graphics Processing Units to Speed Up Subgraph Enumeration for Efficient Graph Pattern Mining GraphDuMato

Subgraph selection involves searching an input graph for subgraphs with a certain attribute. Graph pattern mining (GPM) relies on this technique, despite its computing complexity and rapid growth. Problems with uncoalesced access to memory, separation, and strain unbalance make efficient subgraph identification on GPUs a huge problem, given GPUs' success in speeding up operations across multiple industries. It is surprising that these challenges have not received sufficient attention in earlier research. This work presents new methods for effectively building and running subgraph enumeration on GPUs. Optimization of computational resource usage is achieved by combining a warp-centric design with a Depth first search (DFS) style approach. Memory efficiency, execution divergence, and GPU activity parallelization could all be enhanced by integrating these two methods. An affordable load balancing level is also incorporated for the purpose of dispersing work among thread warps, which further decreases GPU idleness. The GraphDuMato system facilitates the utilization of GPM methods with its intuitive application programming interface (API). Testing proves that GraphDuMato can outperform state-of-the-art GPM algorithms on a regular basis and may mine subgraphs with up to twelve trees.

Efficient Prototyping of a Field-Programmable Gate Array-Based Real-Time Model of a Modular Multilevel Converter

Field-programmable gate array (FPGA)-based real-time simulation plays a crucial role in testing power–electronic dominated systems with the formation of controller hardware-in-the-loop (CHIL) or power hardware-in-the-loop (PHIL). This work describes an efficient implementation of computation time and resource usage in the FPGA-based study of a modular multilevel converter (MMC) with detailed electromagnetic transients. The proposed modeling technique can be used in continuous control mode (CCM) and discontinuous control mode (DCM) for high-switching frequency semiconductor technologies. An FPGA-based designed solver structure is also presented to take advantage of the parallel features of FPGAs to achieve an ultra-fast calculation speed. In addition, two different switch modeling techniques are discussed with a five-level MMC case study. Experimental results on the NI PXIe platform show the feasibility of the proposed implementation, and a time step of 100 nanoseconds is achieved.

Pangenome databases improve host removal and mycobacteria classification from clinical metagenomic data.

Culture-free real-time sequencing of clinical metagenomic samples promises both rapid pathogen detection and antimicrobial resistance profiling. However, this approach introduces the risk of patient DNA leakage. To mitigate this risk, we need near-comprehensive removal of human DNA sequences at the point of sequencing, typically involving the use of resource-constrained devices. Existing benchmarks have largely focused on the use of standardized databases and largely ignored the computational requirements of depletion pipelines as well as the impact of human genome diversity. We benchmarked host removal pipelines on simulated and artificial real Illumina and Nanopore metagenomic samples. We found that construction of a custom kraken database containing diverse human genomes results in the best balance of accuracy and computational resource usage. In addition, we benchmarked pipelines using kraken and minimap2 for taxonomic classification of Mycobacterium reads using standard and custom databases. With a database representative of the Mycobacterium genus, both tools obtained improved specificity and sensitivity, compared to the standard databases for classification of Mycobacterium tuberculosis. Computational efficiency of these custom databases was superior to most standard approaches, allowing them to be executed on a laptop device. Customized pangenome databases provide the best balance of accuracy and computational efficiency when compared to standard databases for the task of human read removal and M. tuberculosis read classification from metagenomic samples. Such databases allow for execution on a laptop, without sacrificing accuracy, an especially important consideration in low-resource settings. We make all customized databases and pipelines freely available.

Seq2seq model for human action recognition based on skeleton and two-layer bidirectional LSTM

Human action recognition (HAR) plays an important role in social interaction in various fields. This study proposes a light-weight skeleton and two-layer bidirectional LSTM-based Seq2Seq model (SB2_Seq2Seq) for HAR to trade off recognition accuracy, users’ privacy and computer resource usage. An experiment was conducted to compare the proposed SB2_Seq2Seq with other skeleton-based Seq2Seq models and non-skeleton RGB video frame-based LSTM, CNN and seq2seq models. The UCF50 dataset was used for model evaluation, where 60%, 20% and 20% for model training, validation and testing, respectively. The experimental results show that the proposed model achieves 93.54% accuracy with 0.0214 Mean Square Error (MSE), suggesting that the proposed model outperforms all the other models. Besides, it also shows that the proposed model achieves state-of-the-art accuracy compared with state-of-the-arts methods in literature.

Low-Light Image Enhancement with Wavelet-Based Diffusion Models

Diffusion models have achieved promising results in image restoration tasks, yet suffer from time-consuming, excessive computational resource consumption, and unstable restoration. To address these issues, we propose a robust and efficient Diffusion-based Low-Light image enhancement approach, dubbed DiffLL. Specifically, we present a wavelet-based conditional diffusion model (WCDM) that leverages the generative power of diffusion models to produce results with satisfactory perceptual fidelity. Additionally, it also takes advantage of the strengths of wavelet transformation to greatly accelerate inference and reduce computational resource usage without sacrificing information. To avoid chaotic content and diversity, we perform both forward diffusion and denoising in the training phase of WCDM, enabling the model to achieve stable denoising and reduce randomness during inference. Moreover, we further design a high-frequency restoration module (HFRM) that utilizes the vertical and horizontal details of the image to complement the diagonal information for better fine-grained restoration. Extensive experiments on publicly available real-world benchmarks demonstrate that our method outperforms the existing state-of-the-art methods both quantitatively and visually, and it achieves remarkable improvements in efficiency compared to previous diffusion-based methods. In addition, we empirically show that the application for low-light face detection also reveals the latent practical values of our method. Code is available at https://github.com/JianghaiSCU/Diffusion-Low-Light.

Collaborative Machine Learning: Schemes, Robustness, and Privacy.

Distributed machine learning (ML) was originally introduced to solve a complex ML problem in a parallel way for more efficient usage of computation resources. In recent years, such learning has been extended to satisfy other objectives, namely, performing learning in situ on the training data at multiple locations and keeping the training datasets private while still allowing sharing of the model. However, these objectives have led to considerable research on the vulnerabilities of distributed learning both in terms of privacy concerns of the training data and the robustness of the learned overall model due to bad or maliciously crafted training data. This article provides a comprehensive survey of various privacy, security, and robustness issues in distributed ML.