File Transfer Service Research Articles

Diabetic foot ulcer photography study: a study within a trial to assess the reliability of two-dimensional (2D) photography for the assessment of ulcer healing in patients with diabetes-related foot ulcers—protocol paper

BackgroundThe primary endpoint in diabetes-related foot ulcer (DFU) trials is often time to healing, defined as complete re-epithelialisation with absence of drainage, requiring clinical expert assessment as the gold standard....

FTS Service Evolution and LHC Run-3 Operations

The File Transfer Service (FTS) is a software system responsible for queuing, scheduling, dispatching and retrying file transfer requests, it is used by three of the LHC experiments, namely ATLAS, CMS and LHCb, as well as non LHC experiments including AMS, Dune and NA62. FTS is critical to the success of many experiments and the service must remain available and performant during the entire LHC Run-3. Experiments use FTS to manage the transfer of their physics files all over the World or more specifically all over the Worldwide LHC Computing Grid (WLCG). Since the start of LHC Run-3 (from 5th July 2022 to 31st August 2023), FTS has managed the successful transfer of approximately 1.2 billion files totalling 1.83 Exabytes of data.This paper describes how the FTS team has evolved the software and the deployment in order to cope with changes in implementation technologies, increase the efficiency of service, streamline its operations, and to meet the ever changing needs of its user community. We report about the software migration from Python 2 to Python 3, the move from the Pylons web development framework towards Flask and the new database deployment strategy to separate the handling of the critical operations from the long duration monitoring queries. In addition, during 2022 a new HTTP based protocol has been finalised that can now be used between FTS and compatible WLCG tape storage endpoints.

NOTED: An intelligent network controller to improve the throughput of large data transfers in File Transfer Services by handling dynamic circuits

The NOTED (Network Optimised Transfer of Experimental Data) project has successfully demonstrated the ability to dynamically provision network links to increase the effective bandwidth available for FTS-driven transfers between endpoints, such as WLCG sites, by inspecting on-going data transfers and so identifying those that are bandwidth-limited for a long period of time. Recently, the architecture of NOTED has been improved and the software has been packaged for easy distribution. These improved capabilities and features of NOTED have been tested and demonstrated at various international conferences. For example, during demonstrations at Supercomputing 2022, independent instances of NOTED at CHCERN (Switzerland) and DE-KIT (Germany) monitored large data transfers generated by the ATLAS experiment between these sites and CA-TRIUMF (Canada). We report here on this and other events, highlighting how NOTED can predict link congestion or a notable increase in the network utilisation over an extended period of time and, where appropriate, automatically reconfigure network topology to introduce an additional or an alternative and betterperforming path by using dynamic circuit provisioning systems such as SENSE and AutoGOLE.

QoE Optimization in a Live Cellular Network through RLC Parameter Tuning.

The mobile communication networks sector has experienced a great evolution during the last few years. The emergence of new services as well as the growth in the number of subscribers have motivated the search for new ways to optimize mobile networks. In this way, the objective pursued by optimization techniques has been evolving, shifting from the traditional optimization of radio parameters to the improvement of the quality perceived by users, known as quality of experience (QoE). In mobile networks, the radio link control (RLC) layer provides a reliable link between both ends of the communication and has a great impact on the QoE. In this paper, the optimization of the QoE for users based on the adjustment of the RLC layer is proposed. For this purpose, two typical services demanded by the users of mobile networks have been selected: the real-time video streaming service and file transfer service. For a broader view of the behavior of the QoE in relation to RLC, optimization tests have been carried out in scenarios with different system bandwidths. In this way, the relationship between the QoE and the optimal configuration of RLC in different network load situations has been analyzed. A proof of concept has been carried out to show the capability of this optimization. To that end, both a cellular network simulator and a live cellular network devised for research purposes have been used.

Are you still printing and posting medical records? If so, why?

GPShare is a fully encrypted, fully secure, NHS approved, file transfer service, saving you over 80%!*

LHC Data Storage: Preparing for the Challenges of Run-3

The CERN IT Storage Group ensures the symbiotic development and operations of storage and data transfer services for all CERN physics data, in particular the data generated by the four LHC experiments (ALICE, ATLAS, CMS and LHCb). In order to accomplish the objectives of the next run of the LHC (Run-3), the Storage Group has undertaken a thorough analysis of the experiments’ requirements, matching them to the appropriate storage and data transfer solutions, and undergoing a rigorous programme of testing to identify and solve any issues before the start of Run-3. In this paper, we present the main challenges presented by each of the four LHC experiments. We describe their workflows, in particular how they communicate with and use the key components provided by the Storage Group: the EOS disk storage system; its archival back-end, the CERN Tape Archive (CTA); and the File Transfer Service (FTS). We also describe the validation and commissioning tests that have been undertaken and challenges overcome: the ATLAS stress tests to push their DAQ system to its limits; the CMS migration from PhEDEx to Rucio, followed by large-scale tests between EOS and CTA with the new FTS “archive monitoring” feature; the LHCb Tier-0 to Tier-1 staging tests and XRootD Third Party Copy (TPC) validation; and the erasure coding performance in ALICE.

NOTED: a framework to optimise network traffic via the analysis of data from File Transfer Services

Network traffic optimisation is difficult as the load is by nature dynamic and seemingly unpredictable. However, the increased usage of file transfer services may help the detection of future loads and the prediction of their expected duration. The NOTED project seeks to do exactly this and to dynamically adapt network topology to deliver improved bandwidth for users of such services. This article introduces, and explains the features of, the two main components of NOTED, the Transfer Broker and the Network Intelligence component. The Transfer Broker analyses all queued and on-going FTS transfers, producing a traffic report which can be used by network controllers. Based on this report and its knowledge of the network topology and routing, the Network Intelligence (NI) component makes decisions as to when a network reconfiguration could be beneficial. Any Software Defined Network controller can then apply these decision to the network, so optimising transfer execution time and reducing operating costs.

FTS3: Data Movement Service in containers deployed in OKD

The File Transfer Service (FTS3) is a data movement service developed at CERN which is used to distribute the majority of the Large Hadron Collider’s data across the Worldwide LHC Computing Grid (WLCG) infrastructure. At Fermilab, we have deployed FTS3 instances for Intensity Frontier experiments (e.g. DUNE) to transfer data in America and Europe, using a container-based strategy. In this article we summarize our experience building docker images based on work from the SLATE project (slateci.io) and deployed in OKD, the community distribution of Red Hat OpenShift. Additionally, we discuss our method of certificate management and maintenance utilizing Kubernetes CronJobs. Finally, we also report on the configuration currently running at Fermilab.

A case study on anonymised sharing platforms and digital traces left by their usage

Non-local forms of file storage and transfer provide investigatory concerns. Whilst mainstream cloud providers offer a well-established challenge to those involved in criminal enquiries, there are also a host of services offering non-account based ‘anonymous’ online temporary file storage and transfer. From the context of a digital forensic investigation, the practitioner examining a suspect device must detect when such services have been utilised by a user, as offending files may not be resident on local storage media. In addition, identifying the use of a service may also expose networks of illegal file distribution, supporting wider investigations into criminal activity. This work examines 16 anonymous file transfer services and identifies and interprets the digital traces left behind on a device following their use to support law enforcement investigations.

On the Capability of QoE Improvement Based on the Adjustment of RLC Parameters.

The constant evolution in mobile communications networks have led operators to seek new techniques to optimize their mobile networks with the objective of satisfying the expectations of the users. In this way, traditional optimization techniques based on improving radio indicators, have given way to new techniques based on improving the quality of experience (QoE) perceived by users. This paper is focused on analyzing the impact of the adjustment of radio link control (RLC) layer configuration parameters on the QoE perceived by the users of two different types of services. Firstly, an evaluation of the QoE experienced by the user of a real-time video streaming service with respect to the transmission buffer size of the RLC layer in unacknowledged mode (UM) has been carried out. Secondly, the QoE perceived by the user of a file transfer service in relation to the variation of the configuration parameters of the RLC layer in acknowledged mode (AM) has been evaluated. The study, which has been carried out in a simulated cellular environment, has been performed for different system bandwidth values, thus proving the relationship between the QoE perceived by the users, the optimal RLC configuration parameters values and the available bandwidth.

FTS improvements for LHC Run-3 and beyond

The File Transfer Service (FTS) developed at CERN and in production since 2014, has become a fundamental component for the LHC experiments and is tightly integrated with experiment frameworks. Starting from the beginning of 2018 with the participation to the European Commission funded project eXtreme Data Cloud (XDC) and the activities carried out in the context of the WLCG DOMA TPC and QoS working groups, a series of new developments and improvements have been planned and performed taking also into account the requirements from the experiments in preparation for the LHC Run-3. This paper provides a detailed overview of these developments; more specifically, the integration with OpenID Connect (OIDC), the QoS integration, the service scalability enhancements, the support for XRootD and HTTP Third Party Copy (TPC) transfers along with the integration with the new CERN Tape Archive (CTA) system.

Anlisis QoS Open IMS Core berbasis Network Function Virtualization pada Protokol TCP

IP Multimedia Subsystem (IMS) is an NGN technology to convergence support of wireline and wireless networks with QoS guarantees. Therefore IMS is widely used by telecommunications operators. Increasing customers make operators overwhelmed by having to replace or add new devices to increase IMS capacity, which is less profitable for operators on the operational costs. The concept of Network Function Virtualization (NFV) is solve the problem. The concept changed of hardware-dedicated to software-dedicated in a virtual environment allows the NFV to be more flexible in increasing device capacity and can reduce dependence on hardware purchases. In this research IMS implemented using Open IMS Core and NFV infrastructure software using OpenStack. analysis based on QoS file transfer services and web servers by measuring parameters throughput, delay, jitter, and packet loss. Based on the measurement of QoS parameters obtained by traffic load of 0 Mbps, 10 Mbps, 20 Mbps, 30Mbps. The throughput average values on file transfer service are 0.2906 Mbit / s and 16.4366 Mbit / s on web server services. The delay average value on file transfer service is 25.5077 ms and 0.7708 ms on the web server service. The resulting jitter value is <1 ms. Percentage of packet loss is < 1%.

CERN Tape Archive — from development to production deployment

The first production version of the CERN Tape Archive (CTA) software is planned to be released during 2019. CTA is designed to replace CASTOR as the CERN tape archive solution, to face the scalability and performance challenges arriving with LHC Run–3. In this paper, we describe the main commonalities and differences between CTA and CASTOR. We outline the functional enhancements and integration steps required to add the CTA tape back-end to an EOS disk storage system. We present and discuss the different deployment and migration scenarios for replacing the five CASTOR instances at CERN, including a description of how the File Transfer Service (FTS) will interface with EOS and CTA.

Internações hospitalares por condições sensíveis à atenção primária à saúde

Objective: To analyze hospital admissions for conditions sensitive to Primary Health Care in the state of Minas Gerais, Brazil. Methods: It is an ecological study with population-based time-series design in the state of Minas Gerais, concerning the hospital admissions and the main causes of them for conditions sensitive to primary care. The analysis was performed based on the Hospitalization Authorization according to the costs, year, gender, and age group based on data from the Hospital Information System (SIH). The micro data were extracted from the file transfer service at the Department of Informatics of the Unified Health System (DATASUS) in June 2018. The records analyzed refer to the period from January 2008 to December 2017, including all hospitalizations for conditions sensitive to primary health care. Results: It was evident that from 2008 to 2017, hospitalizations sensitive to primary care accounted for 21.21% (2,506,782) of the total hospitalizations in the state of Minas Gerais, with Jequitinhonha Region followed by the Northeast and Center South contributing to the higher percentages. Regarding causes, the highest prevalence is concentrated in circulatory system diseases 54.22% (816,617), followed by respiratory system diseases 43.48% (586,809), endocrine diseases 85.70% (325,082), genitourinary diseases 37 , 13% (309,281) and infectious and parasitic diseases 38.37% (266,294). Conclusion: hospital admissions due to conditions sensitive to Primary Health Care in the state of Minas Gerais follow a national pattern, but although hospitalizations are decreasing, it still represents more than 20% of all hospitalizations.

Energy efficient modelling of a network

Most of the networks are generally less energy efficient and most of the time resources are underutilized. Even resources of busy networks are also underutilized and thus networks show energy inefficient management system. This paper focuses on how to obtain resources for the current situation of the network to maintain connectivity, power saving and quality of service. Four different models are proposed in this perspective with different purposes and functions. These models determine the resources under certain constrains. Two types of services namely, minimum bandwidth and trivial file transfer are considered. For minimum bandwidth service, edge, delay and change models are proposed. Here data rate switch and enable/ disable of edges are placed in these models for power saving strategy. Another model, multi flow is proposed for trivial file transfer service. It is proposed for transferring files through multiple flows in multiple paths from source to destination. All models except multi flow model are mixed integer programming optimization problem.

Experiences with the new ATLAS Distributed Data Management System

The ATLAS Distributed Data Management (DDM) system has evolved drastically in the last two years with the Rucio software fully replacing the previous system before the start of LHC Run-2. The ATLAS DDM system manages now more than 250 petabytes spread on 130 storage sites and can handle file transfer rates of up to 30Hz. In this paper, we discuss our experience acquired in developing, commissioning, running and maintaining such a large system. First, we describe the general architecture of the system, our integration with external services like the WLCG File Transfer Service and the evolution of the system over its first years of production. Then, we show the performance of the system, describe the integration of new technologies such as object stores, and outline some new developments, which mainly focus on performance and automation.

Transfer Time, Energy, and Quota-Aware Multi-RAT Operation Scheme in Smartphone

Today's smartphones integrate multiple radio access technologies (multi-RAT), e.g., third-generation (3G), fourth-generation (4G), Wi-Fi, and Bluetooth, whereas the number of integrated RATs is increasing. Therefore, it is becoming more important to select the best RAT set among the available RATs and determine how much data to transfer via each selected RAT network. The authors propose a multi-RAT interface activation algorithm with supporting system design for smartphones' file transfer service (e.g., downloading a movie file). The authors model a multiattribute cost function incorporating file-transfer completion time, energy consumption, and data usage quota together. The goal of this paper is to find out the optimal multi-RAT set to be activated with the corresponding file segment allocation that minimizes the cost function under given energy and quota constraints. By the proposed algorithm, the suboptimal solution can be found without significant performance degradation from the optimal solution while making the computational complexity linear in the number of RATs.

Investigation of the Impact of Ddos Attack on Network Efficiency of the University Of Zakho

In this paper an investigation was conducted to have an insight into the impact of the DDoS attack on the network efficiency of the University of Zakho, in particular, after deploying a fiber infrastructure and a datacenter with services such as web server, email server, and ftp server. Tests were conducted on one of the most popular DDoS attacks, that is, the flooding attack that has many types based on the start-of-the-art research in this field. For the evaluation purpose, two Internet services were chosen which are namely; file transfer service and E-learning video service. OPNET was used as a simulation tool to simulate Zakho University network and to conduct attacks because of its high reliability and reputation in this field. Results showed that DDoS attack has a big adverse impact on the legitimate users’ accessibility for both services. Where the file traffic download reduced from 6000 Bytes/s in the case of no attack into only 500 Bytes/s in the case of attack which makes an approximate of 92% loss in the network traffic. In addition, video throughput is reduced from 180KB/s in the case of no attack into less than 20KB/s in the case of attack which makes around 89% loss that is considered a huge degradation in the network performance.

FTS3: Quantitative Monitoring

The overall success of LHC data processing depends heavily on stable, reliable and fast data distribution. The Worldwide LHC Computing Grid (WLCG) relies on the File Transfer Service (FTS) as the data movement middleware for moving sets of files from one site to another. This paper describes the components of FTS3 monitoring infrastructure and how they are built to satisfy the common and particular requirements of the LHC experiments. We show how the system provides a complete and detailed cross-virtual organization (VO) picture of transfers for sites, operators and VOs. This information has proven critical due to the shared nature of the infrastructure, allowing a complete view of all transfers on shared network links between various workflows and VOs using the same FTS transfer manager. We also report on the performance of the FTS service itself, using data generated by the aforementioned monitoring infrastructure both during the commissioning and the first phase of production. We also explain how this monitoring information and network metrics produced can be used both as a starting point for troubleshooting data transfer issues, but also as a mechanism to collect information such as transfer efficiency between sites, achieved throughput and its evolution over time, most common errors, etc, and take decision upon them to further optimize transfer workflows. The service setup is subject to sites policies to control the network resource usage, as well as all the VOs making use of the Grid resources at the site to satisfy their requirements. FTS3 is the new version of FTS and has been deployed in production in August 2014.

Resilient FTS3 service at GridKa

The FTS (File Transfer Service) service provides a transfer job scheduler to distribute and replicate vast amounts of data over the heterogeneous WLCG infrastructures. Compared to the channel model of the previous versions, the most recent version of FTS simplifies and improves the flexibility of the service while reducing the load to the service components. The improvements allow to handle a higher number of transfers with a single FTS3 setup. Covering now continent-wide transfers compared to the previous version, whose installations handled only transfers within specific clouds, a resilient system becomes even more necessary with the increased number of depending users.Having set up a FTS3 services at the German T1 site GridKa at KIT in Karlsruhe, we present our experiences on the preparations for a high-availability FTS3 service. Trying to avoid single points of failure, we rely on a database cluster as fault tolerant data back-end and the FTS3 service deployed on an own cluster setup to provide a resilient infrastructure for the users. With the database cluster providing a basic resilience for the data back-end, we ensure on the FTS3 service level a consistent and reliable database access through a proxy solution. On each FTS3 node a HAproxy instance is monitoring the integrity of each database node and distributes database queries over the whole cluster for load balancing during normal operations; in case of a broken database node, the proxy excludes it transparently to the local FTS3 service. The FTS3 service itself consists of a main and a backup instance, which takes over the identity of the main instance, i.e., IP, in case of an error using a CTDB (Cluster Trivial Database) infrastructure offering clients a consistent service.