Data Storage Platform Research Articles

DataForge: Modular platform for data storage and analysis

DataForge is a framework for automated data acquisition, storage and analysis based on modern achievements of applied programming. The aim of the DataForge is to automate some standard tasks like parallel data processing, logging, output sorting and distributed computing. Also the framework extensively uses declarative programming principles via meta-data concept which allows a certain degree of meta-programming and improves results reproducibility.

Research and Design of Vital Sign Data Service Platform

Objective: The production of vital sign data is characterized by high concurrency and large amount of data. The data of user's vital sign hashigh medical and economic value, and it is very necessary to store it.Methods: Using a single-threaded, asynchronous I/O, event-driven Node.js and non-relational database MongoDB to build a data platformsuitable for high concurrency and massive data storage of vital signs.Results: By establishing the data storage platform of vital sign data, we can further analyze and utilize the data, and it is of great significanceto develop other health services on this basis, which is of great significance for cultivating users' health care consciousness, preventingdisease and medical research.

Real-time data capture with electronic cigarettes for smoking cessation programme: a cloud platform for behavioural research

Tobacco smoking is a major risk factor for many chronic diseases, which led to the popularity of electronic cigarette (e-cigarette) and showed to be an effective tool for tobacco smoking...

Spatial cumulative sum algorithm with big data analytics for climate change detection

Big data plays a vital role in the prediction of diseases that occur due to climate change. For such predictions, scalable data storage platforms and efficient change detection algorithms are required to monitor the climate change. However, traditional data storage techniques and algorithms are not applicable to process the huge amount of climate data. This paper presents a scalable data processing framework with a novel change detection algorithm. The large volume of climate data is stored on Hadoop Distributed File System (HDFS) and MapReduce algorithm is applied to calculate the seasonal average of climate parameters. Spatial autocorrelation based climate change detection algorithm is proposed in this paper to monitor the changes in the seasonal climate. The proposed climate change detection algorithm is compared with various existing approaches such as pruned exact linear time method, binary segmentation method, and segment neighborhood method.

Cloud computing and big data: Technologies and applications

Cloud computing and big data: Technologies and applications

Converged photonic data storage and switch platform for exascale disaggregated data centers

Converged photonic data storage and switch platform for exascale disaggregated data centers

Improving big data storage performance in hybrid environment

Hybrid storage satisfies the requirements of real-time processing and large capacity put forward by big data. But the widely used big data storage platform HDFS still cannot efficiently utilize emerging devices in hybrid environment. And most of existing hybrid storage researches also fail to consider the asymmetric characteristics among devices and data. This paper proposes a preference model to quantitatively weight the storage performance imbalance when data are distributed on different devices, and then distributes data on storage device whose performance efficiently matches data access characteristics. The implemented Preference-Aware HDFS (PAHDFS) shows high performance, efficiency and scalability in experiments.

A software architecture for monitoring big data storage platforms

A software architecture for monitoring big data storage platforms

The ATLAS Eventlndex: data flow and inclusion of other metadata

The ATLAS EventIndex is the catalogue of the event-related metadata for the information collected from the ATLAS detector. The basic unit of this information is the event record, containing the event identification parameters, pointers to the files containing this event as well as trigger decision information. The main use case for the EventIndex is event picking, as well as data consistency checks for large production campaigns. The EventIndex employs the Hadoop platform for data storage and handling, as well as a messaging system for the collection of information. The information for the EventIndex is collected both at Tier-0, when the data are first produced, and from the Grid, when various types of derived data are produced. The EventIndex uses various types of auxiliary information from other ATLAS sources for data collection and processing: trigger tables from the condition metadata database (COMA), dataset information from the data catalogue AMI and the Rucio data management system and information on production jobs from the ATLAS production system. The ATLAS production system is also used for the collection of event information from the Grid jobs. EventIndex developments started in 2012 and in the middle of 2015 the system was commissioned and started collecting event metadata, as a part of ATLAS Distributed Computing operations.

Highly efficient valence state switching of samarium in BaFCl:Sm nanocrystals in the deep UV for multilevel optical data storage

We report on the highly efficient generation of stable Sm2+ in nanocrystalline BaFCl:Sm3+ by exposure in the deep UV (UV-C) below 250 nm. The generated Sm2+ can be read out via its distinct fluorescence signature which is efficiently excited around 425 nm by the relatively strong f-d transitions with e ≈400 (l mol−1 cm−1). The generation of Sm2+ can also be reversed, erasing the fluorescence signal via two-photon ionization by increasing the power at 425 nm from ≤ 1 mW/cm2 to ~100 mW/cm2. It follows that the switching mechanism is based on oxygen impurities that are in close proximity to the Sm3+ ions. The photoionization kinetics indicate that the average Sm3+- oxide impurity separation is a few interionic spacings. The level of Sm3+- Sm2+ conversion in BaFCl is shown to be tunable over a large dynamic range, and therefore could serve as a platform for rewritable ultra-high density multi-level optical data storage. The present study also sheds light on BaFCl:Sm3+ as a photoluminescent X-ray storage phosphor.

A proof-of-concept evaluation of a cloud-based store-and-forward telemedicine app for screening for oral diseases

Objective It is widely considered that telemedicine can make positive contributions to dental practice. This study aimed to evaluate a cloud-based telemedicine application for screening for oral diseases. Methods A telemedicine system, based on a store-and-forward method, was developed to work as a platform for data storage. An Android application was developed to facilitate entering demographic details and capturing oral photos. As a proof-of-concept, six volunteers were enrolled in a trial to obtain oral images using smartphone cameras. Following an onsite oral examination, images of participants’ teeth were obtained by a trained dental assistant. Oral images were directly uploaded from the smartphone to a cloud-based server via broadband network. The assessments of oral images by offsite dentists were compared with those carried out via face-to-face oral examinations. Results A complete set of 30 oral images was obtained from all six participants. Out of 192 teeth reviewed, the proportion of ungradable teeth was 8%. Sensitivity and specificity of teledental screening were 57% and 100% respectively. The inter-grader agreement estimated for two examination modalities and between two teledental graders was 70% and 62% respectively. Findings indicate that the proposed system for screening of oral diseases can be implemented to provide a valid and reliable alternative to traditional oral screening. Conclusion This study provided evidence that a robust system for store-and-forward screening for dental problems can be developed, and leads to the need for further testing of its robustness to confirm the accuracy and reliability of the teledentistry system.

Mechanistic Modeling of a Rewritable Recombinase Addressable Data Module.

Many of the most important applications predicted to arise from Synthetic Biology will require engineered cellular memory with the capability to store data in a rewritable and reversible manner upon induction by transient stimuli. DNA recombination provides an ideal platform for cellular data storage and has allowed the development of a rewritable recombinase addressable data (RAD) module, capable of efficient data storage within a chromosome. Here, we develop the first detailed mechanistic model of DNA recombination, and validate it against a new set of in vitro data on recombination efficiencies across a range of different concentrations of integrase and gp3. Investigation of in vivo recombination dynamics using our model reveals the importance of fully accounting for all mechanistic features of DNA recombination in order to accurately predict the effect of different switching strategies on RAD module performance, and highlights its usefulness as a design tool for building future synthetic circuitry.

Discovering biomarkers for antidepressant response: protocol from the Canadian biomarker integration network in depression (CAN-BIND) and clinical characteristics of the first patient cohort.

BackgroundMajor Depressive Disorder (MDD) is among the most prevalent and disabling medical conditions worldwide. Identification of clinical and biological markers (“biomarkers”) of treatment response could personalize clinical decisions and lead to better outcomes. This paper describes the aims, design, and methods of a discovery study of biomarkers in antidepressant treatment response, conducted by the Canadian Biomarker Integration Network in Depression (CAN-BIND). The CAN-BIND research program investigates and identifies biomarkers that help to predict outcomes in patients with MDD treated with antidepressant medication. The primary objective of this initial study (known as CAN-BIND-1) is to identify individual and integrated neuroimaging, electrophysiological, molecular, and clinical predictors of response to sequential antidepressant monotherapy and adjunctive therapy in MDD.MethodsCAN-BIND-1 is a multisite initiative involving 6 academic health centres working collaboratively with other universities and research centres. In the 16-week protocol, patients with MDD are treated with a first-line antidepressant (escitalopram 10–20 mg/d) that, if clinically warranted after eight weeks, is augmented with an evidence-based, add-on medication (aripiprazole 2–10 mg/d). Comprehensive datasets are obtained using clinical rating scales; behavioural, dimensional, and functioning/quality of life measures; neurocognitive testing; genomic, genetic, and proteomic profiling from blood samples; combined structural and functional magnetic resonance imaging; and electroencephalography. De-identified data from all sites are aggregated within a secure neuroinformatics platform for data integration, management, storage, and analyses. Statistical analyses will include multivariate and machine-learning techniques to identify predictors, moderators, and mediators of treatment response.DiscussionFrom June 2013 to February 2015, a cohort of 134 participants (85 outpatients with MDD and 49 healthy participants) has been evaluated at baseline. The clinical characteristics of this cohort are similar to other studies of MDD. Recruitment at all sites is ongoing to a target sample of 290 participants. CAN-BIND will identify biomarkers of treatment response in MDD through extensive clinical, molecular, and imaging assessments, in order to improve treatment practice and clinical outcomes. It will also create an innovative, robust platform and database for future research.Trial registrationClinicalTrials.gov identifier NCT01655706. Registered July 27, 2012.

Challenges of big data storage and management

The amount of data generated daily by industries, large organizations and research institute is increasing at a very fast rate. These huge volumes of data need to be kept not just for analytic purposes, but also in compliance with laws and service level agreements to protect and preserve data. Storage and management are major concern in this era of big data. The ability for storage devices to scale to meet the rate of data growth, enhance access time and data transfer rate is equally challenging. These factors, to a considerable extent, determine the overall performance of data storage and management. Big data storage requirements are complex and thus needs a holistic approach to mitigate its challenges. This paper examines the challenges of big data storage and management. In addition, we also examines existing current big data storage and management platforms and provide useful suggestions in mitigating these challenges.Keywords: big data, storage systems, challenges, performance.

Large Optical Backplane With Embedded Graded-Index Glass Waveguides and Fiber-Flex Termination

Introduction of electro-optical circuit board (EOCB) technologies based on embedded glass waveguides in the system enclosure of current data storage, compute, or switch platforms will be instrumental in accommodating the prohibitive bandwidth densities projected in exascale data centers, access networks, and high-performance computing environments in future. The main focus of this paper is, therefore, on the realization of large EOCB backplanes (350 × 465) mm2 with passive dual-star interconnect topologies. The waveguides are embedded inside glass panels using a two-step thermal ion exchange process [1] . The fabricated embedded waveguides are then characterized for propagation and coupling losses across different wavelength ranges and for different coupling arrangements. The fabrication process for these large panels is briefly explained accompanied by the lamination process of four separate glass panels. The connectivity between backplane and peripheral cards is addressed using fiber flex termination with two possible waveguide interfaces—actively assembled pluggable interface and passively assembled adhesive bonded interface. A demonstration platform is also finally reported and the viability evaluated for optical connectivity and system efficiency is tested for the whole system containing fiber flex termination.

Chado use case: storing genomic, genetic and breeding data of Rosaceae and Gossypium crops in Chado.

The Genome Database for Rosaceae (GDR) and CottonGen are comprehensive online data repositories that provide access to integrated genomic, genetic and breeding data through search, visualization and analysis tools for Rosaceae crops and Gossypium (cotton). These online databases use Chado, an open-source, generic and ontology-driven database schema for biological data, as the primary data storage platform. Chado is highly normalized and uses ontologies to indicate the ‘types’ of data. Therefore, Chado is flexible such that it has been used to house genomic, genetic and breeding data for GDR and CottonGen. These data include whole genome sequence and annotation, transcripts, molecular markers, genetic maps, Quantitative Trait Loci, Mendelian Trait Loci, traits, germplasm, pedigrees, large scale phenotypic and genotypic data, ontologies and publications. We provide information about how to store these types of data in Chado using GDR and CottonGen as examples sites that were converted from an older legacy infrastructure. Database URL: GDR (www.rosaceae.org), CottonGen (www.cottongen.org)

Mobile Public e-Government Cloud Services Platform

The mobile public e-Government cloud services platform solution will encompass new technologies, and through the Big Data concept for data collection and exchange it will enable innovative channels of communication and collaboration among different players. The major benefit of the cloud-based platform is the efficient execution of heavy computation algorithms in the cloud simply by using Big Data storage and processing platforms. This new approach will address the mismatch between labour demand and supply to make the services of the public employment agencies more efficient, more effective, user-friendly and personalized. The main focus is given on the positive impact of e-services for employability prospects of young unemployed people, those with disabilities, and older workers. The platform will facilitate the matching of people for job openings, and more specifically, the job search by people with special skill-profiles, including special needs or time requirements. It will provide a networking platform and information services to address socio-economic side-effects of labour market exclusion. The main purpose of the model of mobile public e-Government cloud services platform is to provide an integrated environment where public institutions will receive complete data analysis.

Design and manufacturing model of customized hydrostatic bearing system based on cloud and big data technology

Hydrostatic systems are considered as essential supporting structures in heavy machine tools. The calculations and analyses for hydrostatic bearings are always laborious because of the involvement of several disciplines such as elastic mechanics, hydromechanics, thermodynamics, and other factors in the design. It is well known that large data and cloud technology are capable of processing and transmission of hydrostatic system studies. In this work, a cloud manufacture model is presented to provide big data storage, transmission, and processing platform for designers, manufacturers, and users of hydrostatic bearings. All participants involved in the manufacturing were linked together by timely information communication in order to ensure that the customized products met with the actual conditions in the cloud server. Based on the actual requirements of the user, the carrying capacity of hydrostatic system was analyzed by a designer using finite difference method and the results were sent to the manufacturer for machining of components. The monitored data from the consumer could be fed back to the designer for performance evaluation through the cloud server. It was found that customized manufacturing tasks could be finished more reliably and efficiently if all participants exchanged the data through cloud platform in real time.

Model of Cloud-Based Services for Data Mining Analysis

<p class="zhengwen"><span lang="EN-GB">New cloud-based services are being developed constantly in order to meet the need for faster, reliable and scalable methods for knowledge discovery. The major benefit of the cloud-based services is the efficient execution of heavy computation algorithms in the cloud simply by using Big Data storage and processing platforms. Therefore, we have proposed a model that provides data mining techniques as cloud-based services that are available to users on their demand. The widely known data mining algorithms have been implemented as Map/Reduce jobs that are been executed as services in cloud architecture. The user simply chooses or uploads the dataset to the cloud, makes appropriate settings for the data mining algorithm, executes the job request to be processed and receives the results. The major benefit of this model of cloud-based services is the efficient execution of heavy computation data mining algorithm in the cloud simply by using the Ankus - Open Source Big Data Mining Tool and StarfishHadoop Log Analyzer. The expected outcome of this research is to offer the integration of the cloud-based services for data mining analysis in order to provide researchers with reliable collaborative data mining analysis model.<strong></strong></span></p>

A New Scheme for Monitoring and Diagnosis of Multistage Manufacturing Processes Using Product Quality Measurements


 
 
 The development of robust monitoring systems for assuring the consistency and stability of multistage manufacturing processes necessitates the use of add-on sensors and advanced data collection, storage, and analysis platforms to deal with the high-dimensional data collected from machines and products in multiple stages. In many cases, such an approach may not be feasible due to high implementation costs and the challenges of obtaining the process parameters and analyzing them effectively. This paper proposes an alternative approach for health monitoring and diagnosis of multistage manufacturing processes based on product quality measurements in a sensor-less environment. In the presented work, the available data consists of product quality parameters measured from multiple product types along with the manufacturing route associated with each product. A Gamma distribution is fit to the data for each parameter within a moving time window. Using the distribution fits, a metric is developed to represent the performance of each machine in a stage compared to its peers producing the same product. This metric is then aggregated across all the products produced by the machine to generate the final metric reflecting the overall performance of the machine. This performance metric is first calculated for the machines in the last stage. After flagging the underperforming machines in the last stage, the samples from those machines are removed from the data set and the remaining samples are used to calculate the similar metric for the prior stage. The suggested approach assumes the random distribution of products from one stage to the next to facilitate the implementation of a comparison-based approach. This approach is tested on a data set collected from a manufacturing plant. The results demonstrate the effectiveness of such approach for monitoring and diagnosis of multistage manufacturing processes when the data is not available from within the process.