Workflow Tasks Research Articles

Formikoj: A flexible library for data management and processing in geophysics—Application for seismic refraction data

We introduce the open-source library formikoj, which provides a flexible framework for managing and processing geophysical data collected in environmental and engineering investigations. To account for the substantial changes regarding the market shares of operating systems within the last two decades, the library is specifically implemented and tested for cross-platform usage. We illustrate the applicability of the formikoj library for the forward modeling of seismic refraction waveform data with the SeismicWaveformModeler based on a custom subsurface model and survey geometry. We use these synthetic seismic data set to demonstrate the fundamental seismic refraction processing capabilities of the SeismicRefractionManager; thus, illustrating the ability to combine modeling and processing tasks in a single workflow. Based on real 3D field measurements we present the available range of possibilities provided by the formikoj library for the processing of seismic refraction survey data. In particular, we explore different visualization techniques of the seismic traveltime readings to enhance their consistency prior to the inversion with the third-party library pyGIMLi. The low-level access provided by the formikoj library aims at enabling users to implement novel modeling, visualization and processing tools specifically designed for their objectives as well as other geophysical methods.

Chest X-ray Analysis Using Deep Learning Algorithm

Abstract: Automatic recognition of key chest X-ray results to help radiologists with clinical workflow tasks like time-sensitive triage, pneumothorax (CXR) is crucial Case screening and unanticipated discoveries. Deep learning (DL) models have become a promising prediction technique with near-human accuracy, but usually suffer from a lack of explain ability. Medical professionals can treat and diagnose illnesses more precisely thanks to automated picture segmentation and feature analysis. Because diverse equipment provides images with varying image quality, automatic segmentation of medical images is difficult. According to one research,15% of 104 patients with pleural effusion died within 30 days. In this paper, we propose a model for automatic diagnosis of 14 different diseases based on chest radiographs using machine learning algorithms. Chest X-rays offer a non-invasive (perhaps bedside) method for tracking the course of illness. A severity score prediction model for COVID-19 pneumonia on chest radiography is presented in this study.

An enhanced vision transformer with wavelet position embedding for histopathological image classification

Histopathological image classification is a fundamental task in pathological diagnosis workflow. It remains a huge challenge due to the complexity of histopathological images. Recently, hybrid methods combining convolutional neural networks(CNN) with vision transformers(ViT) are proposed to this field. These methods can well represent the global and local contextual information and achieve excellent classification performances. However, the downsampling operation like max-pooling which ignores the sampling theorem transmits the jagged artifacts into transformer, which would lead to an aliasing phenomenon. It makes the subsequent feature maps focus on the incorrect regions and influences the final classification results. In this work, we propose an enhanced vision transformer with wavelet position embedding to tackle this challenge. In particular, a wavelet position embedding module, which introduces the wave transform into position embedding, is employed to enhance the smoothness of discontinuous feature information by decomposing sequences into amplitude and phase in pathological feature maps. In addition, an external multi-head attention is proposed to replace self-attention in the transformer block with two linear layers. It reduces the cost of computation and excavates potential correlations between different samples. We evaluate the proposed method on three public histopathological classification challenging datasets, and perform a quantitative comparison with previous state-of-the-art methods. The results empirically demonstrate that our method achieves the best accuracy. Furthermore, it has the least parameters and a very low FLOPs. In conclusion, the enhanced vision transformer shows high classification performances and demonstrates significant potential for assisting pathologists in pathological diagnosis.

A Workflow Scheduling Approach With Modified Fuzzy Adaptive Genetic Algorithm in IaaS Clouds

The emergence of the cloud platform with substantial resources to offer on-demand instigated the researchers to migrate the scientific workflows to the cloud environment. The scheduling of workflows with diverse QoS parameters is not a trivial task, but an NP-Complete problem. Several heuristics for QoS constrained workflows have been investigated. However, most of them focus only on time and cost and do not guarantee high resource utilization. The scheduling of the workflow tasks over the minimum cloud resources under the defined time limit is a grave concern. In this article, an algorithm named MFGA (Modified Fuzzy Adaptive Genetic Algorithm) has been formulated to minimize the makespan and improve resource utilization under both deadline and budget constraints. A fuzzy logic controller has also been devised to control the crossover and mutation rates that prevent MFGA from getting stuck in a local optimum. MFGA has a novel crossover technique that adds the fittest solutions in the population. Additionally, a new mutation technique has also been introduced, which minimizes the makespan and increases the reusability of the resources. The simulation experiments with the real workflows show that the proposed MFGA outperforms other state-of-the-art algorithms.

Replication-Based Dynamic Energy-Aware Resource Provisioning for Scientific Workflows

Distributed computing services in cloud environments are easily accessible to end users. These services are delivered to end users via a subscription-based model. The “infrastructure as a service” (IaaS) cloud model is one of the best cloud environment models for running data- and computing-intensive applications. Real-world scientific applications are the best examples of data and computing intensiveness. For their implementation, scientific workflow applications need high-performance computational resources and a large volume of storage. The workflow tasks are linked based on computational and data interdependence. Considering the high volume and variety of scientific workflows (SWs), the resources of the IaaS cloud model require managing energy efficiently and without failure or loss. Therefore, in order to address the issues of power consumption and task failure for real-world SWs, this research work proposes a replication-based dynamic energy-aware resource provisioning (R-DEAR) strategy for SWs in an IaaS cloud environment. The proposed strategy, R-DEAR, is a resource- and service-provisioning strategy that implements a replication-based fault-tolerant and load-balancing mechanism. The proposed R-DEAR strategy schedules the tasks of a scientific workflow with a replication-based fault-tolerant mechanism. The proposed R-DEAR strategy also manages the power consumption of IaaS cloud resources dynamically through a load-sharing process. Simulation results show that the proposed R-DEAR strategy reduces energy consumption, execution cost, and execution time by 9%, 15%, and 18%, respectively, as compared with the existing state-of-the-art strategy.

Safety-Critical Task Offloading Heuristics for Workflow Applications in Mobile Edge Computing

As the fundamental mechanism in mobile edge computing (MEC), task offloading strategy is of great significance to the quality of computing services provided by MEC systems. When coping with workflow applications, the precedence relations among tasks increase the difficulty in developing task offloading strategies. This paper studies the problem of safety-critical task offloading for workflow applications in a MEC environment. Considering the precedence constraints on workflow tasks and the overhead of security services, we formulate the safety-critical workflow offloading model with the objective of jointly optimizing the total completion time and energy consumption. By using a task sequence to represent a feasible solution to the optimization model, we introduce a family of heuristics to solve the safety-critical workflow offloading problem under precedence constraints upon workflow tasks. Depending on whether the offloading solution satisfies the precedence relations among workflow tasks, task sequences can be classified into two categories, i.e., precedence-aware and precedence-unaware offloading solutions. With the satisfaction of precedence constraints, a family of heuristics by using a precedence-aware strategy and a precedence-unaware strategy is designed to offload safety-critical workflow tasks. Given an offloading sequence and the operating conditions of MEC servers, the heuristic algorithms select the currently best MEC server to offload workflow tasks. Experimental results justify the performance of the proposed algorithms in solving the safety-critical workflow offloading problem under precedence constraints.

Structure-Aware Scheduling Methods for Scientific Workflows in Cloud

Scientific workflows consist of numerous tasks subject to constraints on data dependency. Effective workflow scheduling is perpetually necessary to efficiently utilize the provided resources to minimize workflow execution cost and time (makespan). Accordingly, cloud computing has emerged as a promising platform for scheduling scientific workflows. In this paper, level- and hierarchy-based scheduling approaches were proposed to address the problem of scheduling scientific workflow in the cloud. In the level-based approach, tasks are partitioned into a set of isolated groups in which available virtual machines (VMs) compete to execute the groups’ tasks. Accordingly, based on a utility function, a task will be assigned to the VM that will achieve the highest utility by executing this task. The hierarchy-based approach employs a look-ahead approach, in which the partitioning of the workflow tasks is performed by considering the entire structure of the workflow, whereby the objective is to reduce the data dependency between the obtained groups. Additionally, in the hierarchy-based approach, a fair-share strategy is employed to determine the share (number of VMs) that will be assigned to each group of tasks. Dividing the available VMs based on the computational requirements of the task groups provides the hierarchy-based approach the advantage of further utilizing the VMs usage. The results show that, on average, both approaches improve the execution time and cost by 27% compared to the benchmarked algorithms.

FMMEHO Based Workflow Scheduling in Virtualized Cloud Environment for Smart Irrigation System

Extensive and exhaustive water utilization for agriculture, industries and ground water consumption for domestic purposes has heavily deterioted the water bodies. Cloud and sensor technology is widely deployed in a several real-time applications, especially in agriculture. The transformation of data obtained from large sensor networks into a valuable knowledge and assests for applications can effectively leverage the techniques like Cloud Computing (CC). In CC, scheduling the workflow is the major concern that focuses on comprehensive execution of workflows without compromising the Quality of Service (QoS). But workflow scheduling augmented with resource allocation is extremely challenging task because of its inherent computational intensity, task dependencies, and heterogeneous cloud resources. In this article, a novel Optimum Energy and Resource Aware Workflow Scheduling (OERES) scheme that is motivated by popular Fuzzy Membership Mutation Elephant Herding Optimization (FMMEHO) algorithm is proposed, that aims to schedule the task workflow to Virtual Machines (VMs) that are involved in computation. This also concentrates on dynamically deploying and un-deploying the VMs pertaining to the task requirements. The FMMEHO algorithm is a popular nature inspired technique, which is rooted on herding patterns of the giant mammals, the elephants. This algorithm employs a clan operator that updates the location and distance of elephants depending resource and energy usage of each clan in the context of matriarch elephant. The proposed OERES schema elevates the resource utilization and simultaneously mitigates the energy usage without compromising the dependency and deadline constraints. This work uses the famous Cloud Sim simulator to simulate the underlying cloud environment to investigate the effectiveness of proposed model. The efficacy of the scheduling methods is examined based on important parameters like mean Resource Utilization (RU), Energy utilization or Consumption/ Task (ECT), Total Energy Consumption (TEC), Makespan and Execution Time per Task (ETT). The results very well portray the effectiveness of proposed OERES algorithm against already existing methods.

Chest X-ray Analysis using Deep Learning

Abstract: Automatic recognition of key chest X-ray results to help radiologists with clinical workflow tasks like time-sensitive triage, pneumothorax (CXR) case screening and unanticipated discoveries. Deep learning models have become a promising prediction technique with near human accuracy, but usually suffer from a lack of explain ability. Medical professionals can treat and diagnose illnesses more precisely using automated picture segmentation and feature analysis. In this paper, we propose a model for automatic diagnosis of 14 different diseases based on chest radiographs using machine learning algorithms. Chest X-rays offer a non-invasive (perhaps bedside) method for tracking the course of illness. A severity score prediction model for COVID-19 pneumonia on chest radiography is presented in this study.

Operations research contribution to totally automated radiotherapy treatment planning: A noncoplanar beam angle and fluence map optimization engine based on optimization models and algorithms

The planning of radiotherapy treatments is based on the use of mathematical optimization models and algorithms. A treatment plan will correspond to an admissible solution to a problem that is defined by the medical prescription. The medical prescription defines the constraints that have to be satisfied, and that are patient dependent. To find a treatment plan complying with all the defined constraints, an objective function is built, having a set of parameters that can be tuned. In the clinical practice, the objective function parameters are tuned through a trial-and-error procedure. One common way of defining this objective function is to consider as parameters weights that are related with the importance of the corresponding structures of interest (volumes to treat and organs to spare), as well as upper and lower bounds that are related with the constraints defined by the medical prescription. Some treatment options, like the set of irradiation directions (beam angles), are usually defined beforehand by the planner, considering previous experiences with similar cases. This trial-and-error process is lengthy, since it can take up to several hours to calculate an admissible treatment plan for a single patient. There have been some research efforts to automate the treatment planning procedure, releasing the planner for other important tasks in the radiotherapy treatment workflow and guaranteeing the consistent calculation of high-quality plans. In this work derivative-free optimization algorithms are integrated with fuzzy inference systems that automatically tune the objective function parameters so that an admissible solution is found. This fully automated approach was tested and assessed considering six head-and-neck cancer cases already treated at the Portuguese Institute of Oncology at Coimbra (IPOC). For the clinical cases tested, comparisons between different treatment plans clearly favor the proposed approach. For a similar tumor coverage, it was possible to improve the sparing of the spinal cord, brainstem and parotids. Automating radiotherapy treatment planning can contribute to improved treatment plans in a consistent way.

Deep learning approach to security enforcement in cloud workflow orchestration

Supporting security and data privacy in cloud workflows has attracted significant research attention. For example, private patients’ data managed by a workflow deployed on the cloud need to be protected, and communication of such data across multiple stakeholders should also be secured. In general, security threats in cloud environments have been studied extensively. Such threats include data breaches, data loss, denial of service, service rejection, and malicious insiders generated from issues such as multi-tenancy, loss of control over data and trust. Supporting the security of a cloud workflow deployed and executed over a dynamic environment, across different platforms, involving different stakeholders, and dynamic data is a difficult task and is the sole responsibility of cloud providers. Therefore, in this paper, we propose an architecture and a formal model for security enforcement in cloud workflow orchestration. The proposed architecture emphasizes monitoring cloud resources, workflow tasks, and the data to detect and predict anomalies in cloud workflow orchestration using a multi-modal approach that combines deep learning, one class classification, and clustering. It also features an adaptation scheme to cope with anomalies and mitigate their effect on the workflow cloud performance. Our prediction model captures unsupervised static and dynamic features as well as reduces the data dimensionality, which leads to better characterization of various cloud workflow tasks, and thus provides better prediction of potential attacks. We conduct a set of experiments to evaluate the proposed anomaly detection, prediction, and adaptation schemes using a real COVID-19 dataset of patient health records. The results of the training and prediction experiments show high anomaly prediction accuracy in terms of precision, recall, and F1 scores. Other experimental results maintained a high execution performance of the cloud workflow after applying adaptation strategy to respond to some detected anomalies. The experiments demonstrate how the proposed architecture prevents unnecessary wastage of resources due to anomaly detection and prediction.

GoFish: A low-cost, open-source platform for closed-loop behavioural experiments on fish

Fish are the most species-rich vertebrate group, displaying vast ecological, anatomical and behavioural diversity, and therefore are of major interest for the study of behaviour and its evolution. However, with respect to other vertebrates, fish are relatively underrepresented in psychological and cognitive research. A greater availability of easily accessible, flexible, open-source experimental platforms that facilitate the automation of task control and data acquisition may help to reduce this bias and improve the scalability and refinement of behavioural experiments in a range of different fish species. Here we present GoFish, a fully automated platform for behavioural experiments in aquatic species. GoFish includes real-time video tracking of subjects, presentation of stimuli in a computer screen, an automatic feeder device, and closed-loop control of task contingencies and data acquisition. The design and software components of the platform are freely available, while the hardware is open-source and relatively inexpensive. The control software, Bonsai, is designed to facilitate rapid development of task workflows and is supported by a growing community of users. As an illustration and test of its use, we present the results of two experiments on discrimination learning, reversal, and choice in goldfish (Carassius auratus). GoFish facilitates the automation of high-throughput protocols and the acquisition of rich behavioural data. Our platform has the potential to become a widely used tool that facilitates complex behavioural experiments in aquatic species.

Improved Harris Hawks Optimization Algorithm Based Data Placement Strategy for Integrated Cloud and Edge Computing

Cloud computing is considered to facilitate a more cost-effective way to deploy scientific workflows. The individual tasks of a scientific workflow necessitate a diversified number of large states that are spatially located in different datacenters, thereby resulting in huge delays during data transmission. Edge computing minimizes the delays in data transmission and supports the fixed storage strategy for scientific workflow private datasets. Therefore, this fixed storage strategy creates huge amount of bottleneck in its storage capacity. At this juncture, integrating the merits of cloud computing and edge computing during the process of rationalizing the data placement of scientific workflows and optimizing the energy and time incurred in data transmission across different datacentres remains a challenge. In this paper, Adaptive Cooperative Foraging and Dispersed Foraging Strategies-Improved Harris Hawks Optimization Algorithm (ACF-DFS-HHOA) is proposed for optimizing the energy and data transmission time in the event of placing data for a specific scientific workflow. This ACF-DFS-HHOA considered the factors influencing transmission delay and energy consumption of data centers into account during the process of rationalizing the data placement of scientific workflows. The adaptive cooperative and dispersed foraging strategy is included in HHOA to guide the position updates that improve population diversity and effectively prevent the algorithm from being trapped into local optimality points. The experimental results of ACF-DFS-HHOA confirmed its predominance in minimizing energy and data transmission time incurred during workflow execution.

On workflow scheduling for latency-sensitive edge computing applications

With the rapid proliferation of edge computing-based solutions, many edge computing applications use the cloud for data processing and analysis. However, latency-sensitive applications have low latency requirements and can be very bandwidth hungry, so processing their collected data through cloud servers is not efficient and cost-effective. For example, traffic management and health condition monitoring applications, require real-time data processing at the network edge to respond immediately to unexpected events. These applications are typically executed as workflows with dependent tasks that require careful scheduling to allocate the appropriate resources to the tasks so that the execution is done in a way that satisfies the users’ target functions. In this work, we evaluate some traditional scheduling heuristics for the execution of workflow tasks in an edge-computing scenario. Our goal is to compare their performance in terms of execution time and cost and show that these heuristics, previously used in scheduling in the context of cloud environments, can also be used in edge computing scenarios. The results show that the MinMin and PSO scheduling algorithms offer the best results with regard to execution time and cost.

Multi-objective scheduling method of workflow task based on tabu search algorithm

Multi-objective scheduling method of workflow task based on tabu search algorithm

Multi-objective Scheduling Method of Workflow Task Based on Tabu Search Algorithm

Multi-objective Scheduling Method of Workflow Task Based on Tabu Search Algorithm

Remote Sensing Data Processing Process Scheduling Based on Reinforcement Learning in Cloud Environment

Task scheduling plays a crucial role in cloud computing and is a key factor determining cloud computing performance. To solve the task scheduling problem for remote sensing data processing in cloud computing, this paper proposes a workflow task scheduling algorithm---Workflow Task Scheduling Algorithm based on Deep Reinforcement Learning (WDRL). The remote sensing data process modeling is transformed into a directed acyclic graph scheduling problem. Then, the algorithm is designed by establishing a Markov decision model and adopting a fitness calculation method. Finally, combine the advantages of reinforcement learning and deep neural networks to minimize make-time for remote sensing data processes from experience. The experiment is based on the development of CloudSim and Python and compares the change of completion time in the process of remote sensing data. The results show that compared with several traditional meta-heuristic scheduling algorithms, WDRL can effectively achieve the goal of optimizing task scheduling efficiency.

Survey on Task Scheduling Optimization Strategy under Multi-Cloud Environment

Cloud computing technology is favored by users because of its strong computing power and convenient services. At the same time, scheduling performance has an extremely efficient impact on promoting carbon neutrality. Currently, scheduling research in the multi-cloud environment aims to address the challenges brought by business demands to cloud data centers during peak hours. Therefore, the scheduling problem has promising application prospects under the multi-cloud environment. This paper points out that the currently studied scheduling problems in the multi-cloud environment mainly include independent task scheduling and workflow task scheduling based on the dependencies between tasks. This paper reviews the concepts, types, objectives, advantages, challenges, and research status of task scheduling in the multi-cloud environment. Task scheduling strategies proposed in the existing related references are analyzed, discussed, and summarized, including research motivation, optimization algorithm, and related objectives. Finally, the research status of the two kinds of task scheduling is compared, and several future important research directions of multi-cloud task scheduling are proposed.

CWL-PLAS: Task Workflows Assisted by Data Science Cloud Platforms

The Common Workflow Language (CWL) is a platform-independent description language for the representation of data science workflows consisting of a set of tasks that interact with each other to perform scientific analysis. The tasks can be packaged as Linux containers. On the one hand, using containers ensures the reproducibility and portability of workflows. Still, on the other hand, it limits each task to exploiting, at most, the resources of the host where its container runs. In this paper, we propose CWL-PLAS, an extension of CWL that allows a task to instantiate and temporarily use a supporting cloud platform for parallel computing, which is specialized for the task’s activity. In this way, tasks can leverage the resources of multiple hosts in parallel, reducing the duration of the workflow. We implemented an open-source workflow manager that supports CWL-PLAS workflows and exploits a Kubernetes back-end. We used this workflow manager to evaluate the performance of CWL-PLAS in a couple of machine learning workflows.

Workflow Scheduling in Cloud Computing Environment by Combining Particle Swarm Optimization and Grey Wolf Optimization

Scheduling workflows is a vital challenge in cloud computing due to its NP-complete nature and if an efficient workflow task scheduling algorithm is not used then it affects the system’s overall performance. Therefore, there is a need for an efficient workflow task scheduling algorithm that can distribute dependent tasks to virtual machines efficiently. In this paper, a hybrid workflow task scheduling algorithm based on a combination of Particle Swarm Optimization and Grey Wolf Optimization (PSO GWO) algorithms, is proposed. PSO GWO overcomes the disadvantages of both PSO and GWO algorithms by improving the exploitation (local search) of PSO algorithm and exploration (global search) of GWO algorithm. This leads to better balance between exploration and exploitation, consequently it minimizes the makespan with 5.52% compared to GWO and 3.68% compared to PSO. The degree of imbalance reduced upto 33.22% compared to GWO and 17.61% compared to PSO, improves the convergence rate as well depending on number tasks and iterations. CloudSim tool is used to evaluate the proposed algorithm. The simulation results confirmed that the proposed method performs better than both of the standard PSO and GWO in terms of makespan, degree of imbalance and convergence rate