Computational Nodes Research Articles

Distributed Aerial Image Stitching on Multiple Processors using Message Passing Interface

This study investigates the potential of using Message Passing Interface (MPI) parallelization to enhance the speed of the image stitching process. The image stitching process involves combining multiple images to create a seamless panoramic view. This research explores the potential benefits of segmenting photos into distributed tasks among several identical processor nodes to expedite the stitching process. However, it is crucial to consider that increasing the number of nodes may introduce a trade-off between the speed and quality of the stitching process. The initial experiments were conducted without MPI, resulting in a stitching time of 1506.63 seconds. Subsequently, the researchers employed MPI parallelization on two computer nodes, which reduced the stitching time to 624 seconds. Further improvement was observed when four computer nodes were used, resulting in a stitching time of 346.8 seconds. These findings highlight the potential benefits of MPI parallelization for image stitching tasks. The reduced stitching time achieved through parallelization demonstrates the ability to accelerate the overall stitching process. However, it is essential to carefully consider the trade-off between speed and quality when determining the optimal number of nodes to employ. By effectively distributing the workload across multiple nodes, researchers and practitioners can take advantage of the parallel processing capabilities offered by MPI to expedite image stitching tasks. Future studies could explore additional optimization techniques and evaluate the impact on speed and quality to achieve an optimal balance in real-world applications.

Photonic Source of Heralded Greenberger-Horne-Zeilinger States.

Generating large multiphoton entangled states is of main interest due to enabling universal photonic quantum computing and all-optical quantum repeater nodes. These applications exploit measurement-based quantum computation using cluster states. Remarkably, it was shown that photonic cluster states of arbitrary size can be generated by using feasible heralded linear optics fusion gates that act on heralded three-photon Greenberger-Horne-Zeilinger (GHZ) states as the initial resource state. Thus, the capability of generating heralded GHZ states is of great importance for scaling up photonic quantum computing. Here, we experimentally demonstrate this required building block by reporting a polarisation-encoded heralded GHZ state of three photons, for which we build a high-rate six-photon source (547±2 Hz) from a solid-state quantum emitter and a stable polarization-based interferometer. The detection of three ancillary photons heralds the generation of three-photon GHZ states among the remaining particles with fidelities up to F=0.7278±0.0106. Our results initiate a path for scalable entangling operations using heralded linear-optics implementations.

Optimizing System Resources and Adaptive Load Balancing Framework Leveraging ACO and Reinforcement Learning Algorithms

In today's constantly changing computer settings, the most important things for improving speed and keeping stability are making the best use of system resources and making sure that load balancing works well. To achieve flexible load balancing and resource optimization, this study suggests a new system that combines the Ant Colony Optimization (ACO) and Reinforcement Learning (RL) methods. The structure is meant to help with the problems that come up when tasks and resource needs change in big spread systems. ACO is based on how ants find food and is used to change how jobs are distributed among computer nodes based on local knowledge and scent tracks. This autonomous method makes it easy to quickly look for solutions and adjust to new situations. In addition to ACO, RL methods are used to learn about and adjust to how the system changes over time. By planning load balancing as a series of decisions, RL agents are able to keep improving their rules so that the system works better and resources are used more efficiently. Agents learn the best ways to divide up tasks and use resources by interacting with the world and getting feedback. The suggested system works in a spread way, which makes it scalable and reliable in a variety of settings. The system changes its behavior on the fly to react to changing tasks and resource availability by using the group intelligence of ACO and the flexibility of RL. The system can also handle different improvement goals and limitations, which makes it flexible and usable in a range of situations. The suggested approach works better than standard load balancing methods at improving system performance, lowering reaction times, and making the best use of resources, as shown by the results of experiments. Using the strengths of the ACO and RL algorithms, this structure looks like a good way to deal with the complexity of current computer systems and make good use of resources in changing settings

FP‐JSC: Job failure prediction on supercomputers through job application sequence correlation

SummarySupercomputers are advanced computing systems interconnected through high‐speed communication networks, consisting of independent computational nodes. During the unfolding of the big data era, the potent computational capabilities of these supercomputers play a pivotal role in scientific computing. Despite executing numerous advanced computational science and engineering tasks on supercomputers, many submitted jobs fail due to various factors, resulting in user inefficiencies. These failures not only consume system resources but also reduce the overall efficiency of the system. Previous research often couples job performance features with a single machine learning method for predicting job failure. However, a primary hurdle emerges from the high cost of gathering these features, complicating their real‐world applicability. To address this challenge, our study establishes correlations among job applications through extensive job log analysis. Leveraging correlations, we propose a predictive framework based on job application sequence correlation (called FP‐JSC). This innovative framework employs multiple machine learning models to offer holistic predictions, selecting the most suitable model based on its learning effectiveness. Moreover, the framework optimizes feature collection expenses without adversely affecting job execution. We determine job applications using both job paths and job names, with the former emerging as a novel feature derived from supplementary monitoring data. Empirical results underscore FP‐JSC's effectiveness, accurately identifying over 89% of jobs with 95% specificity and 89% sensitivity—outperforming single prediction methods employed in related works.

Efficient Full-Frequency GW Calculations Using a Lanczos Method.

The GW approximation is widely used for reliable and accurate modeling of single-particle excitations. It also serves as a starting point for many theoretical methods, such as its use in the Bethe-Salpeter equation (BSE) and dynamical mean-field theory. However, full-frequency GW calculations for large systems with hundreds of atoms remain computationally challenging, even after years of efforts to reduce the prefactor and improve scaling. We propose a method that reformulates the correlation part of the GW self-energy as a resolvent of a Hermitian matrix, which can be efficiently and accurately computed using the standard Lanczos method. This method enables full-frequency GW calculations of material systems with a few hundred atoms on a single computing workstation. We further demonstrate the efficiency of the method by calculating the defect-state energies of silicon quantum dots with diameters up to 4nm and nearly 2,000 silicon atoms using only 20 computational nodes.

A scalable MapReduce-based design of an unsupervised entity resolution system.

Traditional data curation processes typically depend on human intervention. As data volume and variety grow exponentially, organizations are striving to increase efficiency of their data processes by automating manual processes and making them as unsupervised as possible. An additional challenge is to make these unsupervised processes scalable to meet the demands of increased data volume. This paper describes the parallelization of an unsupervised entity resolution (ER) process. ER is a component of many different data curation processes because it clusters records from multiple data sources that refer to the same real-world entity, such as the same customer, patient, or product. The ability to scale ER processes is particularly important because the computation effort of ER increases quadratically with data volume. The Data Washing Machine (DWM) is an already proposed unsupervised ER system which clusters references from diverse data sources. This work aims at solving the single-threaded nature of the DWM by adopting the parallelization nature of Hadoop MapReduce. However, the proposed parallelization method can be applied to both supervised systems, where matching rules are created by experts, and unsupervised systems, where expert intervention is not required. The DWM uses an entropy measure to self-evaluate the quality of record clustering. The current single-threaded implementations of the DWM in Python and Java are not scalable beyond a few 1,000 records and rely on large, shared memory. The objective of this research is to solve the major two shortcomings of the current design of the DWM which are the creation and usage of shared memory and lack of scalability by leveraging on the power of Hadoop MapReduce. We propose Hadoop Data Washing Machine (HDWM), a MapReduce implementation of the legacy DWM. The scalability of the proposed system is displayed using publicly available ER datasets. Based on results from our experiment, we conclude that HDWM can cluster from 1,000's to millions of equivalent references using multiple computational nodes with independent RAM and CPU cores.

Probing an Easy-to-Deploy Multi-Agent Manufacturing System Based on Agent Computing Node: Architecture, Implementation, and Case Study

Abstract Due to the widespread adoption of personalized customization services, the application contexts within discrete manufacturing workshops have become increasingly intricate, necessitating the modern industry to evolve toward a more adaptable production trajectory. The pre-established production rules in a traditional centralized control manufacturing system present difficulties in accommodating dynamic situations. Although a multi-agent manufacturing system (MAMS) yields natural advantages in handling dynamic emergencies, the current research is limited to the computer simulation level and lacks integration with the underlying physical devices. In order to mitigate said challenges, the standardization modeling approach for the agent computing node (ACN) to facilitate the implementation of a readily deployable MAMS was proposed in the present study. Initially, adapters encompassing communication, decision, and control modules were developed within the industrial personnel computer-based computing node to accommodate the heterogeneous interface protocols of diverse machines. These adapters enable communication and interaction among machines while laying the computational foundation for the ACN. Accordingly, the models of the machine agent, the part agent, and the monitoring agent were constructed based on ACNs and could perceive the dynamic production information and support the enabling application. Subsequently, to guide ACNs in making scheduling decisions beneficial to global performance, an improved negotiation mechanism in MAMS was achieved in real-time. Finally, the proposed MAMS based on the ACN was deployed in an actual flexible machining workshop. Comparative experiments were implemented and, as exhibited from the experimental results, the proposed ACNs possessed the capabilities of achieving optimal global decision-making and facilitating straightforward deployment.

Ontological Modeling and Clustering Techniques for Service Allocation on the Edge: A Comprehensive Framework

Nowadays, we are in a world of large amounts of heterogeneous devices with varying computational resources, ranging from small devices to large supercomputers, located on the cloud, edge or other abstraction layers in between. At the same time, software tasks need to be performed. They have specific computational or other types of requirements and must also be executed at a particular physical location. Moreover, both services and devices may change dynamically. In this context, methods are needed to effectively schedule efficient allocations of services to computational resources. In this article, we present a framework to address this problem. Our proposal first uses knowledge graphs for describing software requirements and the availability of resources for services and computing nodes, respectively. To this end, we proposed an ontology that extends our previous work. Then, we proposed a hierarchical filtering approach to decide the best allocation of services to computational nodes. We carried out simulations to evaluate four different clustering strategies. The results showed different performances in terms of the number of allocated services and node overload.

Foggier skies, clearer clouds: A real-time IoT-DDoS attack mitigation framework in fog-assisted software-defined cyber-physical systems

IntroductionThe convergence of the Internet of Anything (IoX), software-defined communication layer, and sophisticated cloud platform has steered the dawn of the fourth industrial revolution era (Industry 4.0). The technological leap forward has given shelter to the umbrella of cyber-physical systems such as Smart Grids, Agriculture, and Healthcare. The exponential growth of vulnerable spots (the Internet of Things) in multi-layer cyber-physical systems is involuntarily contributing to malignant IoT-generated denial-of-service attacks. The IoT devices manifest a high degree of diversity in traffic patterns, and a single dataset is insufficient to cover fiducial attack scenarios. In addition, the research community is pressing to overcome other roadblocks, such as an optimal architecture for solution efficacy, performance issues, and the need for comprehensive validation of the proposed solutions. MethodsThe paper offers a five-stage defense framework for building a mitigation against IoT-based DDoS attacks. The article brings forth an amalgamated dataset concocted from InSDN, BoT-IoT, and UNSW-Sydney datasets and a simulated dataset for IoT-DDoS to the research community. The paper employs a multi-stage Stack-Ensembled framework at the heart of the solution pillared on physical devices' behavioral attributes, resulting in a universal defense approach. The experiment leverages fog computing with distributed computational nodes to reduce response latency. Furthermore, the paper implements attack-detection-as-a-service on top of the Docker framework for a cost-effective, reusable, and portable framework. The novel design of the framework presents a light mitigation scheme to the SDN controller, ensuring a negligible impact on the controller's performance. Results and DiscussionThe hand-crafted feature selection process reduced the features by 80%, demonstrating a high accuracy of 99.99% with benchmark datasets, 98.84% accuracy in the simulation environment, and collateral damage of 1.52%. The experiment observes encouraging values for vital performance parameters that researchers often miss to discuss. Furthermore, the paper thoroughly analyzes IoT-DDoS mitigation framework performance parameters.

TCEC: Integrity Protection for Containers by Trusted Chip on IoT Edge Computing Nodes

TCEC: Integrity Protection for Containers by Trusted Chip on IoT Edge Computing Nodes

Application of automatic driving task serialisation monitoring for operating robots

Abstract With the increasing application of industrial robots, more and more simple and repetitive operations in industrial automated production lines are replaced by industrial robots. In this study, a path planning algorithm is proposed for the autopilot task of an operating robot to find the optimal path from the start point to the goal point under the constraints of safe obstacle avoidance, shortest distance, and fastest time. Forward kinematics is used to detect whether it will collide with environmental obstacles until the first feasible result is detected as the end position for navigation. Serialized monitoring is used to obtain the activity information of each computational node, the hardware state information of the system, and the state of the autopilot task processing. Finally, obstacle avoidance navigation experiments on a mobile operating robot are conducted to verify the effectiveness and feasibility of the obstacle avoidance navigation algorithm. The results show that the maximum error of the autopilot task of the operating robot is 2.91cm, and the average error is 0.13 2cm, which is in line with the trajectory tracking error requirements of the operating machine control method and verifies the validity and practicability of this study.

Creating Distributed Artificial Neural Networks Based on Orthogonal Transformations

The article addresses the issue of separating input information of artificial neural networks into modules using orthogonal transformations. This separation enables modular organization of neural networks with layer separation, facilitating the use of the proposed approach for distributed computing. Such an approach is required for organizing the operation of neural networks in fog and edge computing environments, as well as for high-performance computing across multiple low-performance computational nodes. The possibility of cross-layer separation of artificial neural networks using orthogonal transformations is theoretically substantiated, and practical examples of such an approach are provided. A comparison of the characteristics of modular neural networks using various types of orthogonal transformations, including the Haar wavelet transform, is conducted.

Platelet activation near point-like source of agonist: Experimental insights and computational model.

Disorders of hemostasis resulting in bleeding or thrombosis are leading cause of mortality in the world. Blood platelets are main players in hemostasis, providing the primary response to the vessel wall injury. In this case, they rapidly switch to the activated state in reaction to the exposed chemical substances such as ADP, collagen and thrombin. Molecular mechanisms of platelet activation are known, and detailed computational models are available. However, they are too complicated for large-scale problems (e.g. simulation of the thrombus growth) where less detailed models are required, which still should take into account the variation of agonist concentration and heterogeneity of platelets. In this paper, we present a simple model of the platelet population response to a spatially inhomogeneous stimulus. First, computational nodes modeling platelets are placed randomly in space. Each platelet is assigned the specific threshold for agonist, which determines whether it becomes activated at a given time. The distribution of the threshold value in a population is assumed to be log-normal. The model was validated against experimental data in a specially designed system, where the photorelease of ADP was caused by localized laser stimulus. In this system, a concentration of ADP obeys 2-dimensional Gaussian distribution which broadens due to the diffusion. The response of platelets to the point-like source of ADP is successfully described by the presented model. Our results advance the understanding of platelet function during hemostatic response. The simulation approach can be incorporated into larger computational models of thrombus formation.

Distributed multi-objective optimization for SNP-SNP interaction detection

The detection of complex interactions between single nucleotide polymorphisms (SNPs) plays a vital role in genome-wide association analysis (GWAS). The multi-objective evolutionary algorithm is a promising technique for SNP-SNP interaction detection. However, as the scale of SNP data further increases, the exponentially growing search space gradually becomes the dominant factor, causing evolutionary algorithm (EA)-based approaches to fall into local optima. In addition, multi-objective genetic operations consume significant amounts of time and computational resources. To this end, this study proposes a distributed multi-objective evolutionary framework (DM-EF) to identify SNP-SNP interactions on large-scale datasets. DM-EF first partitions the entire search space into several subspaces based on a space-partitioning strategy, which is nondestructive because it guarantees that each feasible solution is assigned to a specific subspace. Thereafter, each subspace is optimized using a multi-objective EA optimizer, and all subspaces are optimized in parallel. A decomposition-based multi-objective firework optimizer (DCFWA) with several problem-guided operators was designed. Finally, the final output is selected from the Pareto-optimal solutions in the historical search of each subspace. DM-EF avoids the preference for a single objective function, handles the heavy computational burden, and enhances the diversity of the population to avoid local optima. Notably, DM-EF is load-balanced and scalable because it can flexibly partition the space according to the number of available computational nodes and problem size. Experiments on both artificial and real-world datasets demonstrate that the proposed method significantly improves the search speed and accuracy.

Large-Scale Meta-Heuristic Feature Selection Based on BPSO Assisted Rough Hypercuboid Approach.

The selection of prominent features for building more compact and efficient models is an important data preprocessing task in the field of data mining. The rough hypercuboid approach is an emerging technique that can be applied to eliminate irrelevant and redundant features, especially for the inexactness problem in approximate numerical classification. By integrating the meta-heuristic-based evolutionary search technique, a novel global search method for numerical feature selection is proposed in this article based on the hybridization of the rough hypercuboid approach and binary particle swarm optimization (BPSO) algorithm, namely RH-BPSO. To further alleviate the issue of high computational cost when processing large-scale datasets, parallelization approaches for calculating the hybrid feature evaluation criteria are presented by decomposing and recombining hypercuboid equivalence partition matrix via horizontal data partitioning. A distributed meta-heuristic optimized rough hypercuboid feature selection (DiRH-BPSO) algorithm is thus developed and embedded in the Apache Spark cloud computing model. Extensive experimental results indicate that RH-BPSO is promising and can significantly outperform the other representative feature selection algorithms in terms of classification accuracy, the cardinality of the selected feature subset, and execution efficiency. Moreover, experiments on distributed-memory multicore clusters show that DiRH-BPSO is significantly faster than its sequential counterpart and is perfectly capable of completing large-scale feature selection tasks that fail on a single node due to memory constraints. Parallel scalability and extensibility analysis also demonstrate that DiRH-BPSO could scale out and extend well with the growth of computational nodes and the volume of data.

Memoization based priority-aware task management for QoS provisioning in IoT gateways

Fog computing is a paradigm that works in tandem with cloud computing. The emergence of fog computing has boosted cloud-based computation, especially in the case of delay-sensitive tasks, as the fog is situated closer to end devices such as sensors that generate data. While scheduling tasks, the fundamental issue is allocating resources to the fog nodes. With the ever-growing demands of the industry, there is a constant need for gateways for efficient task offloading and resource allocation, for improving the Quality of Service (QoS) parameters. This paper focuses on the smart gateways to enhance QoS and proposes a smart gateway framework for delay-sensitive and computation-intensive tasks. The proposed framework has been divided into two phases: task scheduling and task offloading. For the task scheduling phase, a dynamic priority-aware task scheduling algorithm (DP-TSA) is proposed to schedule the incoming task based on their priorities. A Memoization based Best-Fit approach (MBFA) algorithm is proposed to offload the task to the selected computational node for the task offloading phase. The proposed framework has been simulated and compared with the traditional baseline algorithms in different test case scenarios. The results show that the proposed framework not only optimized latency and throughput but also reduced energy consumption and was scalable as against the traditional algorithms.

Navigating the Landscape of Distributed File Systems: Architectures, Implementations, and Considerations

Distributed File Systems (DFS) have emerged as sophisticated solutions for efficient file storage and management across interconnected computer nodes. The main objective of DFS is to achieve flexible, scalable, and resilient file storage management by dispersing file data across multiple interconnected computer nodes, enabling users to seamlessly access and manipulate files distributed across diverse nodes. This article provides an overview of DFS, its architecture, classification methods, design considerations, challenges, and common implementations. Common DFS implementations discussed include NFS, AFS, GFS, HDFS, and CephFS, each tailored to specific use cases and design goals. Understanding the nuances of DFS architecture, classification, and design considerations is crucial for developing efficient, stable, and secure distributed file systems to meet diverse user and application needs in modern computing environments.

Improving Numerical Accuracy of the Localized Oscillatory Radial Basis Functions Collocation Method for Solving Elliptic Partial Differential Equations in 2D

Recently, the localized oscillatory radial basis functions collocation method (L-ORBFs) has been introduced to solve elliptic partial differential equations in 2D with a large number of computational nodes. The research clearly shows that the L-ORBFs is very convenient and useful for solving large-scale problems, but this method is numerically less accurate. In this paper, we propose a numerical scheme to improve the accuracy of the L-ORBFs by adding low-degree polynomials in the localized collocation process. The numerical results validate that the proposed numerical scheme is highly accurate and clearly outperforms the results of the L-ORBFs.

EasyDock: customizable and scalable docking tool

Docking of large compound collections becomes an important procedure to discover new chemical entities. Screening of large sets of compounds may also occur in de novo design projects guided by molecular docking. To facilitate these processes, there is a need for automated tools capable of efficiently docking a large number of molecules using multiple computational nodes within a reasonable timeframe. These tools should also allow for easy integration of new docking programs and provide a user-friendly program interface to support the development of further approaches utilizing docking as a foundation. Currently available tools have certain limitations, such as lacking a convenient program interface or lacking support for distributed computations. In response to these limitations, we have developed a module called EasyDock. It can be deployed over a network of computational nodes using the Dask library, without requiring a specific cluster scheduler. Furthermore, we have proposed and implemented a simple model that predicts the runtime of docking experiments and applied it to minimize overall docking time. The current version of EasyDock supports popular docking programs, namely Autodock Vina, gnina, and smina. Additionally, we implemented a supplementary feature to enable docking of boron-containing compounds, which are not inherently supported by Vina and smina, and demonstrated its applicability on a set of 55 PDB protein-ligand complexes.

Research on distributed coding computation based on matrix multiplication

Matrix multiplication is used in machine learning to train larger, more accurate models with massive data more effectively. The distributed machine learning paradigm necessitates the distributed computing of many matrix multiplications once it is adopted. The execution time of distributed machine learning algorithms increases on dropout nodes, which are computational nodes in distributed clusters that randomly slow down computation due to a variety of factors (e.g., node failure, system failure, communication bottlenecks, etc.), becoming a significant bottleneck in distributed computing systems. It has been discovered that coded computation is less expensive than replica methods and can more effectively reduce the effects of dropped nodes. In this study, we present theoretical insights on how encoded solutions might produce significant gains over unencoded solutions.