CSV Files Research Articles

CUSTOMER SENTIMENT ANALYSIS THROUGH SOCIAL MEDIA FEEDBACK: A CASE STUDY ON TELECOMMUNICATION COMPANY

Customer sentiment analysis is an automated way of detecting sentiments in online interactions in order to assess customer opinions about a product, brand or service. It assists companies in gaining insights and efficiently responding to their customers. This study presents a machine learning approach to analyse how sentiment analysis detects positive and negative feedback about a telecommunication company’s products. Customer feedback data were taken from Twitter through Streaming API (Application Programming Interface), where Tweets are retrieved in real time based on search terms, time, users and likes. Responses from the twitter API are parsed into tables and stored in a CSV file. Based on the analysis, it was found that there was no negative sentiment from the customers. The data were then split into training and testing to be tested on the three different supervised learning algorithms used in this study which are Support Vector Machine, Random Forest, and Naïve Bayes. Lasty, the performance of each model was compared to select the most accurate model and from the analysis, it can be concluded that Support Vector Machine gives the best performance in terms of accuracy, Mean Squared Error, Root Mean Squared Error and Area Under the ROC curve.

Investigation of aerosol droplets diameter generated in aerosol jet printing

Abstract Aerosol jet printing is a contactless direct-write technique that could be used for the deposition of a variety of materials. First, used for electric paths, the technology was explored for many applications. The substantial part of the process is the generation of aerosols. The size of the droplets and the stability of the process affect the quality of the sprayed lines. This article investigates the diameter of the sprayed droplets, allowing future comparison of the results with sprayed lines. Droplets from ultrasonic and pneumatic generators were sprayed at their outlet on the polyethylene terephthalate (PET) foil. Using a digital microscope and the built-in algorithm, the diameter of the droplets was measured, and the dataset was collected as CSV files and served as a background to the box plot. Scanning electron microscopy (SEM) and atomic force microscopy (AFM) scans were applied to verify the results obtained. The ink parameters used in the process have an influence on the aerosol generation and droplet diameter, whereas the carrier gas pressure has an impact mostly on the droplet diameter. In this case, the aerosol was produced from three types of ink in combination with two generators. For inks with a dynamic viscosity below 6.5 m·Pa−1·s−1 a stable range of 5–10 μm droplet diameter was observed. A high-viscosity ink (7.5–10.5 m·Pa−1·s−1) produced droplets with diameter in the range of 6–25 μm. The diameter of the droplet decreased from 7–22 μm to 1–5 μm with a reduction in the dynamic viscosity from 7.5–10.5 m·Pa−1·s−1 to 4.5–5.5 m·Pa−1·s−1.

The PATCH Lab v1.0: A database and workspace for Cenozoic terrestrial paleoclimate and environment reconstruction

In the last two decades, analytical advances and a growing interest in relevant research questions has brought a rapid increase in the amount of stable isotope data used for reconstructing terrestrial paleoclimates and environments. As the spatial and temporal resolution of proxy data continues to improve, the quantitative interpretation of these data is becoming increasingly common. These advances in data resolution and theory bring opportunities for multi-proxy comparisons, synthesis and modeling of large datasets, integration with paleoecological datasets, improved climate model benchmarking, and more. Here, in an effort to support these growing avenues of research, we present <i>The PATCH Lab</i> (Paleo-Analysis of Terrestrial Climate and Hydrology)—an online portal to discover, download, and quantitatively analyze deep time (<math xmlns="http://www.w3.org/1998/Math/MathML" id="u1" display="inline" overflow="scroll"><mrow><mo>&gt;</mo><mn>1</mn></mrow></math> Ma) terrestrial stable isotope data. The PATCH Lab portal hosts a new database that currently includes 27009 stable isotope measurements from 211 publications spanning multiple terrestrial proxies, and quantitative models for interpreting water isotope and soil carbonate data. Data query, download, and modeling results are organized into user-friendly graphical interfaces that export datasets as .csv files. New data can be easily submitted to the PATCH Lab curators through the portal by completing a data submission template. The PATCH Lab, with the help of community engagement, serves as a resource for archiving terrestrial stable isotope data, building paleo “isoscapes”, and increasing accessibility to quantitative methods of investigating terrestrial stable isotopes in paleoclimate.

Ontology-based semantic retrieval of documents using Word2vec model

Semantic retrieval of engineering knowledge is crucial in various engineering activities, such as process development and product model planning. To make a solution for this issue, previously, a few word-based semantic enabling existing approaches such as Lexical Retrieval Model with Semantic Residual Embedding’s (LR-SRE), Document Retrieval Model through Semantic Linking (DR-MTS), Semantic Term Matching in Axiomatic Approaches to Information Retrieval (STM-IR) and Hybrid Ontology for Semantic Information Retrieval Model using Keyword Matching Indexing System (HOS-IR) were utilized. But, they all have shown less query-based accuracy results than the required value. In our proposed Information Retrieval (IR) design, the semantic knowledge-based retrieval scheme has been implemented. For query, entered by a user and processed for finding the dominated word. Word is then compared for its similarity equations, and similarity values are then computed to give output. Highly similar values are obtained as the class value. From class, the respective clusters are selected. Then, documents in that cluster are retrieved and ranked according to the relevance of the user. To support the accuracy level performance of this IR system, a word2vec model has been employed with the benefits of Horse Herd Optimization (HHO), which helps to extract vectors as features for classification. These results are stored as a .csv file for further retrieval. By implementing the proposed IR-word2vec model, the results showed that it outperforms other existing techniques by improved similarity index and accuracy for query results in an execution time of 1.7 s.

Enabling FAIR data in Earth and environmental science with community-centric (meta)data reporting formats

Research can be more transparent and collaborative by using Findable, Accessible, Interoperable, and Reusable (FAIR) principles to publish Earth and environmental science data. Reporting formats—instructions, templates, and tools for consistently formatting data within a discipline—can help make data more accessible and reusable. However, the immense diversity of data types across Earth science disciplines makes development and adoption challenging. Here, we describe 11 community reporting formats for a diverse set of Earth science (meta)data including cross-domain metadata (dataset metadata, location metadata, sample metadata), file-formatting guidelines (file-level metadata, CSV files, terrestrial model data archiving), and domain-specific reporting formats for some biological, geochemical, and hydrological data (amplicon abundance tables, leaf-level gas exchange, soil respiration, water and sediment chemistry, sensor-based hydrologic measurements). More broadly, we provide guidelines that communities can use to create new (meta)data formats that integrate with their scientific workflows. Such reporting formats have the potential to accelerate scientific discovery and predictions by making it easier for data contributors to provide (meta)data that are more interoperable and reusable.

Development of Real-Time Landmark-Based Emotion Recognition CNN for Masked Faces.

Owing to the availability of a wide range of emotion recognition applications in our lives, such as for mental status calculation, the demand for high-performance emotion recognition approaches remains uncertain. Nevertheless, the wearing of facial masks has been indispensable during the COVID-19 pandemic. In this study, we propose a graph-based emotion recognition method that adopts landmarks on the upper part of the face. Based on the proposed approach, several pre-processing steps were applied. After pre-processing, facial expression features need to be extracted from facial key points. The main steps of emotion recognition on masked faces include face detection by using Haar-Cascade, landmark implementation through a media-pipe face mesh model, and model training on seven emotional classes. The FER-2013 dataset was used for model training. An emotion detection model was developed for non-masked faces. Thereafter, landmarks were applied to the upper part of the face. After the detection of faces and landmark locations were extracted, we captured coordinates of emotional class landmarks and exported to a comma-separated values (csv) file. After that, model weights were transferred to the emotional classes. Finally, a landmark-based emotion recognition model for the upper facial parts was tested both on images and in real time using a web camera application. The results showed that the proposed model achieved an overall accuracy of 91.2% for seven emotional classes in the case of an image application. Image based emotion detection of the proposed model accuracy showed relatively higher results than the real-time emotion detection.

Automatic User Preferences Selection of Smart Hearing Aid Using BioAid.

Noisy environments, changes and variations in the volume of speech, and non-face-to-face conversations impair the user experience with hearing aids. Generally, a hearing aid amplifies sounds so that a hearing-impaired person can listen, converse, and actively engage in daily activities. Presently, there are some sophisticated hearing aid algorithms available that operate on numerous frequency bands to not only amplify but also provide tuning and noise filtering to minimize background distractions. One of those is the BioAid assistive hearing system, which is an open-source, freely available downloadable app with twenty-four tuning settings. Critically, with this device, a person suffering with hearing loss must manually alter the settings/tuning of their hearing device when their surroundings and scene changes in order to attain a comfortable level of hearing. However, this manual switching among multiple tuning settings is inconvenient and cumbersome since the user is forced to switch to the state that best matches the scene every time the auditory environment changes. The goal of this study is to eliminate this manual switching and automate the BioAid with a scene classification algorithm so that the system automatically identifies the user-selected preferences based on adequate training. The aim of acoustic scene classification is to recognize the audio signature of one of the predefined scene classes that best represent the environment in which it was recorded. BioAid, an open-source biological inspired hearing aid algorithm, is used after conversion to Python. The proposed method consists of two main parts: classification of auditory scenes and selection of hearing aid tuning settings based on user experiences. The DCASE2017 dataset is utilized for scene classification. Among the many classifiers that were trained and tested, random forests have the highest accuracy of 99.7%. In the second part, clean speech audios from the LJ speech dataset are combined with scenes, and the user is asked to listen to the resulting audios and adjust the presets and subsets. A CSV file stores the selection of presets and subsets at which the user can hear clearly against the scenes. Various classifiers are trained on the dataset of user preferences. After training, clean speech audio was convolved with the scene and fed as input to the scene classifier that predicts the scene. The predicted scene was then fed as input to the preset classifier that predicts the user’s choice for preset and subset. The BioAid is automatically tuned to the predicted selection. The accuracy of random forest in the prediction of presets and subsets was 100%. This proposed approach has great potential to eliminate the tedious manual switching of hearing assistive device parameters by allowing hearing-impaired individuals to actively participate in daily life by automatically adjusting hearing aid settings based on the acoustic scene.

FOCUS ON ARTIFICIAL INTELLIGENCE FOR PREDICTING THE OUTFLOW OF CLIENTS FROM ON-LINE EDUCATION SITES

The article examines the task of assessing the cost of housing in the cities of Ukraine. The purpose of this work is to simplify the determination of the value of apartments on the real estate market using machine learning technologies. To solve this problem, it is proposed to use a program module in Python using the Sequential direct distribution model of the keras library. A program was created that estimates the value of apartments according to their parameters using a neural network. The importance of forecasting in the field of real estate is shown, because the housing market is a systemic part of the regional economy. The results of the software application, which consists of two parts, are presented. The first program collects the necessary data for training a neural network about apartments from the OLX site ads, their structuring and recording in a csv file. The second program provides tools for preliminary analysis of the collected data, after which they are cleaned, divided into training and test samples and trained on their basis by a multilayer neural network of direct propagation using a machine learning algorithm. The layers API of the keras library was used to design the neural network, which allows the user to create arbitrary layers. For regularization, the keras.regularizers tool, which is also in the layers API, is used. To configure model metrics, the compile method was used. Three hidden layers were defined, for each of which 512 neurons were introduced and the Relu activation function was chosen. Calculations of the correlation of prediction indicators and error curves of machine learning are given. As a result of testing the trained neural network on a test set of 652 examples, an average absolute error of 3570.88 was obtained, and the accuracy of the model was approximately 85%. Thus, the neural network has reached an acceptable level of accuracy for estimating the cost of apartments in the city of Kharkiv. Ways to reduce test errors and learning errors using cross-validation are proposed. Concepts of learning hyper-parameters and their regularization are considered Keywords: neural networks, deep learning, machine learning, regression, prediction, estimation, data analysis.

DYNAMIC AND WEB-BASED 4D VISUALIZATION OF STREETSPACE ACTIVITIES DERIVED FROM TRAFFIC SIMULATIONS AND SEMANTIC 3D CITY MODELS

Abstract. Semantic 3D city models can serve as anchor points for different components of urban digital twins. In addition to static 3D models such as buildings, transportation infrastructure, vegetation, or city furniture, this can also include dynamic processes such as traffic movement or changing traffic signals. Integrating these aspects into a dynamic, realistic, and accessible 4D visualization presents a number of requirements and challenges, which are discussed. While the City Geography Markup Language (CityGML) is a well established OGC standard for modeling and exchanging semantic 3D city models, the Cesium Language (CZML) provides capabilities for visualizing time-dependant properties that can be displayed in the Cesium virtual globe. Results of the open-source microscopic traffic simulation tool SUMO include information on locations and orientations of vehicles, bicycles, pedestrians and other traffic members as well as traffic signal information and can be exported as a CSV table. In order to allow a 4D visualization of these simulation results, a scheme for deriving 3D + t CZML documents from data given as 2D + t CSV files is presented. Additionally, dynamically changing traffic lights are integrated. Based on data available for the city of Munich, a CityGMLcompliant streetspace model is generated. This enables the combination of traffic simulation results and semantic 3D city models within a common Cesium based web-visualization and allows a direct and platform independent access to realistic visualizations of streetspace activities. Since this often results in a huge amount of data, a spatio-temporal tiling strategy enabling the visualization of large CZML data is presented.

Application of elements of the information and analytical system of crop management to ensure resource-saving production of flax seeds in the Tver region

Relevance. The concept of precision agriculture call for the use of physical and mathematical, technical and software tools for obtaining and processing information about agroecosystems, as well as for the implementation of agricultural practices directly in the field. To date, there are quite a few software products that can control the production of crops, in particular, the management of differentiated fertilization and compliance with elements of cultivation technology. The purpose of our research conducted over two years was to develop and test in real field conditions algorithms for the functioning of the module of the information and analytical system of crop management for specialized crop rotations involving flax.Methods. The objects of research were the elements of the information and analytical management system for resource-saving production of crop production "Vash urozhaj" (IASUR) and flax-long-lived variety Tonus (selection of the FSBI FNC LC). The experiment used an electronic field map in kml file format containing the coordinates of points, quantitative characteristics of indicators of the content of nutrients and chemical elements in elementary areas within the surveyed spatial object, a technological map of the cultivation of flax, developed specifically for this variety, task maps for the application of solid and liquid fertilizers in .csv file format.Results. Developed and formed in the information and analytical system "Vash urozhaj" electronic technological map of the cultivation of flax-long-legged variety Tonus, containing a list of technological operations, timing and quality characteristics of their implementation, including a possible set of machinery and equipment. The possibility has been tested and the effectiveness of the module for obtaining climatically secured flax yield has been determined — the planned estimated yield for this flax variety at the pilot site is 4.3 c/ha. When cultivated according to the technology adopted in the Federal Scientific Center of Bast Cultures, introduced into the agroplatform, a yield of 4.0 c/ha was obtained. The yield deviation by 6.98% relative to the calculated one is due to the weather features of the growing season of 2021.

ECG Paper Digitization and R Peaks Detection Using FFT

An electrocardiogram (ECG) uses electrodes to monitor the heart rhythm and identify minute electrical changes that occur with each beat. It is employed to investigate particular varieties of aberrant heart activity, such as arrhythmias and conduction problems. One of the most essential tools for detecting heart problems is the electrocardiogram (ECG). The majority of ECG records are still on paper. Manual ECG paper record analysis can be difficult and time-consuming. It is possible to digitally digitize these paper ECG recordings for automated analysis and diagnosis. In this paper, we proposed a system to digitize the ECG paper, automatically detecting R peaks, calculating the average heart rate, and sending SMS to the doctor via cloud in the event of detection of abnormality. The method of the system is uploading an ECG image, then dimensionality reduction, feature extraction in the form of digital signals, and saving it in a CSV file format using the MATLAB programming language. After that, the system retrieves the signals for further processing of the raw signals. We used the fast Fourier transform (FFT) algorithm to calculate R peaks and calculate the heart rate. If the heart rate is abnormal, the system sends SMS messages to doctors via a technology platform (Twilio) using the Python programming language.

Mental, Physical, and Cognitive Wellbeing during the COVID-19 Pandemic: Data from Scotland and Japan

We present data from two studies examining how COVID-19 restrictions affected health behaviours (alcohol consumption, diet, sleep quality, and physical activity levels), mental wellbeing (negative mood) and cognitive function (decision making, attention, learning, working memory, and time perception) in association with socio-demographic factors. Study 1 assessed participants in Scotland and presents cognitive function data for five timepoints. Study 2 is transnational, assessing participants in Scotland and Japan. Data are stored as CSV files. Reuse may involve examining further effects of pandemic enforced social isolation or serve as baseline data when assessing social isolation in expeditions or ageing.

New Software for DQE Calculation in Digital Mammography Compliant with IEC 62220–1-2

Significant improvements in mammography systems have been achieved with the introduction of active matrix flat-panel digital detectors. The advent of this technology also makes it possible to implement computational methods for quantitative image analysis. This study describes new software created to perform detective quantum efficiency (DQE) calculations fully compliant with the IEC 62220–1-2 standard. Python-based software was developed that contains modules to calculate inverse conversion function, modulation transfer function (MTF), noise power spectrum (NPS), and DQE itself. A graphical user interface (GUI) and further add-ons make this software more user-friendly. Results are immediately displayed diagrammatically, and complete output data are exported to a .csv file. The code is available freely, as a compiled, executable file (.exe). The program was successfully tested using DICOM images obtained from mammography units from different manufacturers. This study also includes validation of the new software, based on comparisons of results obtained for the same set of data with two other, freely available programs.

Developments, trends, and publication opportunities in sports medicine: Bibliometric analysis

The development of publications in sports medicine has never been mapped, what themes are trending from year to year, and there are opportunities for interesting themes to be studied in the future. This study aims to determine the trend of publications in the field of sports medicine in Indonesia until 2022. This study was a quantitative study with a bibliometric analysis approach using Vosviewers software using keywords co-occurrence approach on secondary published bibliographic data from the Scopus database and manual exploration of distribution frequency of some variables in the CSV file. The search results found that 260 articles were published in both journals and proceedings indexed by Scopus from 1994 to 2022. The findings showed that there had been an increase in scientific production in the last five years. The University of Indonesia, with a total of 27 documents, became one of the affiliates that contributed the most documents on sports medicine in the Scopus database, while one of the most productive authors was Andri Maruli Tua Lubis, with a total of 5 documents. IOP Conference Series Materials Science and Engineering were the dominant source of documents in the field of sports medicine, with 19 documents. The mapping indicates that injury prevention and increasing physical activity are among the major clusters of research in the area of sports medicine. Finally, the bibliographic analysis also indicated that the integration of sports medicine with technology is still the direction of the future.

An Application of IoT in a Drone Inspection Service for Environmental Control

This paper presents an exploratory activity with a drone inspection service for environmental control. The aim of the service is to provide technical support to decision-makers in environmental risk management. The proposed service uses IoT for the interaction between a mobile application, a Smart City platform, and an Unmanned Aircraft System (UAS). The mobile application allows the users to report risky situations, such as fire ignition, spills of pollutants in water, or illegal dumping; the user has only to specify the class of the event, while the geographical coordinates are automatically taken from device-integrated GPS. The message sent from the mobile application arrives to a Smart City platform, which shows all the received alerts on a 3D satellite map, to support decision-makers in choosing where a drone inspection is required. From the Smart City platform, the message is sent to the drone service operator; a CSV file defining the itinerary of the drone is automatically built and shown through the platform; the drone starts the mission providing a video, which is used by the decision-makers to understand whether the situation calls for immediate action. An experimental activity in an open field was carried out to validate the whole chain, from the alert to the drone mission, enriched by a Smart City platform to enable a decision-maker to better manage the situation.

FIREBASE EFFICIENCY IN CSV DATA EXCHANGE THROUGH PHP-BASED WEBSITES

A database is a collection of data that can be classified into two types: SQL (structure query language), which is associated with relational databases, and NoSQL (non-relational SQL), which is associated with distributed databases. As the world becomes increasingly technologically advanced and computerized, the amount of information grows exponentially, resulting in data. In the majority of studies, the NoSQL database is referred to as "NoSQL." Firebase, a NoSQL database, is used to exchange data with a CSV file on a php-based website in this study. Two CSV files of 1000 and 4997 records were used for both the importing and exporting processes to collect the experimental results. Firebase's speed was examined by exchanging csv files with a php-based website in this research.

DESIGN METHODOLOGY LEXICO-SEMANTIC GRAPHS WITH THE HELP OF THE GEPHI COMPUTER PROGRAM

The article examines the problems of searching various formal models that are able to reflect fragments of the surrounding reality as structures of consciousness of speakers of a certain language. Graphs were established to be the most suitable for formalizing knowledge about fragments of existence among such models. The construction of graphs reproduces the relationship between the components of these fragments. The author made an excursion into the theory of graphs, characterized their typology and revealed possibilities of using graphs in linguistic studies, in particular for the formalization of elements of the phonological, morphological, syntactic and lexico-semantic language levels.The developed methodical algorithm for constructing a graph of the lexico-semantic field of the pandemic using the Gephi computer programme helped to visualize the structure of the lexico-semantic field and the relationship between its core and periphery, as well as to measure the degree of semantic density of the elements of the lexico-semantic field. The research showed the following advantages of the selected computer programme: support for the Windows operating system; simplicity and user friendliness; possibility to upload the necessary document to the system (CSV file), which facilitates the user’s work, because it is not necessary to manually enter data about vertices and edges into the programme; preview of the constructed graph and editing of its elements; saving in a special format for further display of the interactive graph on the Internet. The selected programme was used to construct a complex undirected lexico-semantic graph with a vertex pandemic.

Reddit Dataset on Meme Stock: GameStop

This dataset includes one-year Reddit posts, post metadata, post sentiments, 57 post meta-features and post comments threads from several subreddits around a meme stock: GameStop. These subreddits are dedicated to the discussion of GameStop stock and the social movement of global wealth transfer that the event GameStop stock short squeeze initiated in January 2021. The subreddits included are r/GME, r/superstonk, r/DDintoGME, and r/GME Jungle. The whole dataset is stored in eight CSV files and four HTML files.

UNITE Species Hypotheses Matching Analysis

UNITE (User-friendly Nordic ITS Ectomycorrhizal Database) (Nilsson et al. 2018) is a theoretical and practical platform for calculating, identifying and communicating DNA-based species hypotheses that may not have been described as formal species yet. UNITE species hypotheses (SH) matching analysis is a digital service for the global species discovery from environmental DNA (eDNA). SH matching service is based on UNITE datasets hosted in PlutoF. The tool places a user's unknown DNA sequences in FASTA format into existing UNITE species hypotheses and forms SHs not yet present in the system. Its output in a series of CSV and HTML files (Fig. 1) includes information about what species are present in eDNA samples, whether they are potentially undescribed new species, where they are found in other studies, whether they are alien or threatened species, etc. The service is very useful for understanding species distribution patterns, host range, etc. Analysis output will provide Digital Object Identifier (DOI)-based stable identifiers for communicating species hypotheses found in eDNA. DOIs are connected to the taxonomic backbone of PlutoF and GBIF (Global Biodiversity Information Facility). In this way every DOI is accompanied by a taxon name, which is still widely used for the communication of species. In the case of undescribed species, DOIs will be issued by the PlutoF system. All UNITE services are focused on fungi but cover all Eukarya by using publicly available rDNA ITS (ribosomal DNA Internal Transcribed Spacer) marker sequences accompanied by sample metadata. SH matching analysis results can be published in GBIF as DNA-derived occurrence data where occurrences are linked to SH identifiers present in GBIF backbone taxonomy. Source code of the SH matching analysis tool is available in GitHub with an implementation accessible online on the PlutoF platform for registered users. PlutoF (Abarenkov et al. 2010) is an open data management platform for biology and related disciplines. Users can manage their bodiversity datasets through a full data life cycle—from uploading to publishing and archiving in FAIR (Findable, Accessible, Interoperable, Reusable) way. PlutoF also features an analysis module by providing analytical services for molecular sequence identification and species discovery from eDNA samples. There are more than 8 000 registrered users in PlutoF with more than 500 users who have used its analysis module (as of August, 2022).

A Pre-ingestion Framework for Darwin Core Archives

The Atlas of Living Australia’s (ALA) Pre-ingestion Framework is our alternative to managing datasets via the Global Biodiversity Information Facility's (GBIF) Integrated Publishing Toolkit (IPT). The framework uses a system-agnostic Python codebase to create and update Darwin Core archives: building an archive from a core and extension csv files, merging two archives together, deleting records and identifying duplicates based on the identifiers. The framework dynamicly supports current Darwin Core and GBIF namespace terms. Previously, this functionality was handled internally by a Java-based biocache-store ingestion application. While flexible and easy to call, this black box approach to data management created challenges like removing problem records and tracking and verifying data sources. Last year, as the ALA merged our ingestion codebase with GBIF's pipelines and upgraded our data store infrastructure, we took the opportunity to manage our source data exclusively as full Darwin Core archives, rather than partial text files or spreadsheets. Consequently, the Python-based framework consolidates a lot of work previously managed using a range of methodologies and technologies including Talend, Java and unix based scripting. Alongside the Darwin Core archive manipulation tools, it has handlers for harvesting data from secure external web services, web hosts or file servers. The standardised approach to data loading paves the way for improved automation and workflow. The work has the potential to become an open source project to share with the Living Atlas and biodiversity informatics communities.