Data Processing Requirements Research Articles

Development of portable user-interactive holographic information collector for agricultural product markets

The efficient and accurate collection of agricultural product market information serves as the basis for the effective regulation of agricultural product markets. To achieve a comprehensive, accurate, and timely collection of agricultural product market information, this study puts forth a technique for collecting holographic information of agricultural product markets. A portable human-machine interaction device with an Advanced RISC (reduced instruction set computer) Machines (ARM)-based processor as the core is developed. Holographic information such as agricultural product market trading time, trading place, product name, price, and trading volume can be collected. Via embedded technology and component technology, innovative agricultural product market positioning, and matching, standardized collection, and data processing, and in combination with intelligent algorithms for analysis and early warning, a mobile application terminal for information collection is developed, namely, a holographic information collector for agricultural product markets (named Nongxincai). By using a layered structure, the hardware integrates microprocessor, storage, power, application and communication interface, and human-machine interaction modules. The device has the advantages of miniaturization, the whole-machine power consumption of less than 0.5 W, and continuous operating time of at least 10 h. A supporting software system for Nongxincai has also been developed, in which Microsoft Windows Mobile 6.5 is employed as the operating system, and the CPU frequency is up to 600 MHz. This configuration fully meets the computing requirements of map processing and large-volume data processing and has high compatibility. Nongxincai has been popularized and applied in 12 provinces/municipalities in China. It has played an important role in the monitoring and early warning of different varieties of agricultural products and target prices of soybean and cotton. Keywords: agricultural product market, holographic information, portable, user-interactive, information collector, monitoring and early warning DOI: 10.25165/j.ijabe.20201303.5845 Citation: Xu S W, Li D H, Zhang Y E, Chen W, Zhuang J Y, Liu J J, et al. Development of portable user-interactive holographic information collector for agricultural product markets. Int J Agric & Biol Eng, 2020; 13(3): 143–153.

Real-time abnormal light curve detection based on a Gated Recurrent Unit network

Targeting the problem of high real-time requirements in astronomical data processing, this paper proposes a real-time early warning model for light curves based on a Gated Recurrent Unit (GRU) network. Using the memory function of the GRU network, a prediction model of the light curve is established, and the model is trained using the collected light curve data, so that the model can predict a star magnitude value for the next moment based on historical star magnitude data. In this paper,we calculate the difference between the model prediction value and the actual observation value and set a threshold. If the difference exceeds the set threshold, the observation value at the next moment is considered to be an abnormal value, and a warning is given. Astronomers can carry out further certification based on the early warning and in combinationwith other means of observation. Themethod proposed in this paper can be applied to real-time observations in time domain astronomy.

Fog-cloud task scheduling of energy consumption optimisation with deadline consideration

The emerging IoT introduces many new challenges that cannot be adequately addressed by the current 'cloud-only' architectures. The cooperation of the fog and cloud is considered to be a promising architecture, which efficiently handles IoT's data processing and communications requirements. However, how to schedule tasks to better adapt to IoT real-time needs and reduce the energy in the fog-cloud system is not well addressed. In this paper, we first model the energy consumption of the fog and cloud, respectively, and formulate a task scheduling problem into a constrained optimisation problem in fog-cloud computing system. Then, an efficient deadline-energy scheduling algorithm based on ant colony optimisation (DEACO) is put forward to tackle this problem, which achieves to reduce energy consumption on the condition of satisfying the task deadline. Finally, algorithms have been simulated on the extended CloudSim simulator. The experimental results have shown that our scheduling approach reduces energy more effective.

Bioinformatics online support for bioactive substances cytotoxicity testing and their statistical analysis

Preclinical in vitro/in vivo testing is the first step in discovery of anticancer medicines, among others evaluation of cytotoxic activity of bioactive substances (BAS) on various human normal and cancer cell lines. Cytotoxicity expressed as IC50 value (a dose that inhibits 50% of cell growth) is one of the most commonly used parameter for comparable analysis of activity between different substances. This study includes examination of number of BAS and their results for cytotoxic activity obtained in the Laboratory for cell and molecular biology (LCMB) that require various statistical and computational techniques for proper effective analysis. In order to improve experimental data analysis, make it faster, more effective, error proof with secure online data repository, a Web application with LCMB IC50 database was developed as a useful research tool acting as a leverage for scientific data processing requirements. Analysis includes cytotoxic effects of chemical and natural BAS (IC50 values) on HCT-116, SW-480, MDA-MB-231, and MRC-5 cell lines. Generally, it can be concluded that BAS of different origin, chemical or natural, have various cytotoxic effects and cause different cell line sensitivity, which is presented and discussed. This paper presents developed SQL database-centric Web application with remote user-friendly data management for a biology researcher user type profile. Data processing in this article can be useful for a further overlook and testing of cytotoxic substances.

PCN518 ONCOLOGY REAL WORLD DATA (RWD): ENABLING LEARNING HEALTH SYSTEMS

Unprecedented advances in oncology innovation and precision medicine are increasing treatment complexity. Sharing Real-World Data (RWD) on routine practice systematically is critical to help navigate this. This abstract aims to describe how this goal can be realized. It draws lessons from a collaborative European data-sharing platform developed to inform best practice, highlight variations in care, catalyze research and help address financial sustainability challenges. Through a collaborative approach with patients, clinicians, biopharmaceutical companies and cancer organizations, IQVIA, supported by the Collaboration for Oncology Data in Europe (CODE), assessed the infrastructural and data requirements of a data-sharing platform to meet oncology stakeholders’ needs. Data protection authorities were consulted to understand data processing and warehouse requirements. Methodologies to address all requirements were then identified. Pre-requisites identified included: low latency data access, comparability, sufficient clinical depth, support and involvement of, and minimal administrative burden to the clinical community, GDPR alignment, and flexibility to facilitate different payment models. The Oncology Data Network (ODN) was constructed using ‘data protection by design,’ aligning with the EU GDPR. The ODN will provide analytically ready data in <24 hours on a concise set of variables including clinical & demographic characteristics, biomarkers, indication of use, and detailed drug utilization including treatment plan, dispensing and administration for all anti-cancer medicines longitudinally in a patients’ medical pathway across settings of care. 115 European sites in 6 countries have joined, representing approximately 80,000 patients receiving anti-cancer therapy. The mapping of 28,000 distinct anti-cancer regimens enables comparability. RWD from the ODN will enable us to generate timely insights and assess utilization of anti-cancer medicines, prescribing patterns, adherence to guidelines, bench marking data across countries, and potentially develop innovative agreements to ultimately improve access for patients.

Benchmarking big data systems: A survey

With the enormous growth on the availability and usage of Big Data storage and processing systems, it has become essential to assess the various performance aspects of these systems so that we can carefully understand their strong and weak aspects. In practice, currently, when an individual/enterprise aims to develop a Big Data storage and processing solution for harnessing the knowledge inside their data, they will get challenged by the availability of several frameworks from which they need to select. This is a challenging task which needs to directed by with good knowledge about various perspectives of such systems. Additionally, the choice normally vary from one scenario to another according to the essential needs of the application. In practice, there is no single benchmark study which can cover the different types of big data processing requirements, systems, application scenarios and metrics. Several benchmarks and benchmarking studies have been developed where each study focuses on some representative type of frameworks and only consider some aspects to cover. In this article, we provide a comprehensive survey and analysis of the state-of-the-art of benchmarking the different types of big data systems (e.g., NoSQL databases, Big SQL engines, Big Streaming engines, Big Graph Processing engines, Big Machine/Deep Learning engines). Additionally, we highlight some of the significant open challenges and missing requirements of current benchmarks of big data systems with suggestions of directions for future extensions and improvements.

Hyperspectral Image Classification via Compressive Sensing

Although hyperspectral technology has continued to improve over the years, it is still limited to size, weight, and power (SWaP) constraints. One major issue is the need to sample a large number of very fine spectral bands. Such prohibitively large size of hyperpsectral data creates challenges in both data archival and processing. Compressive sensing (CS) is an enabling technology for reducing the overall data processing and SWaP requirements. This paper explores the viability of performing classification for hyperspectral data on a compressively sensed band domain (CSBD) via CS instead of the original data space, without performing sparse reconstruction. In particular, the well-known restricted isometry property (RIP) and a random spectral sampling strategy are explored for hyperspectral image classification (HSIC) in CSBD. A mathematical analysis is also presented to show that the classification error can be expressed in terms of the restricted isometry constant (RIC) so that the HSIC in the original full-band data space can be achieved in CSBD provided that sufficient band-sensing conditions are met. To validate the proposed CS-HSIC a set of real hyperspectral image experiments are performed where a commonly used spectral–spatial classification algorithm based on support vector machine (SVM) and edge-preserving filters (EPFs) is implemented for a comparative study and analysis. The results clearly demonstrate the potential of CS-HSIC in future research directions.

Big Data para ayudar a generar alertas tempranas en salud pública. Diseño de una arquitectura de software para sistemas Big Data

The massive production of data in different formats and sources, governmental, social, and legal has created the possibility that government institutions in México can have a clear vision of what society thinks about specific issues. In public health, these data are the base for generating alert indicators on outbreaks of diseases in various regions or communities, based on epidemiological intelligence concepts. The problem that institutions face is the lack of an architecture of software systems suitable for collect, catalog and analyze for to take better decisions and action routes to the health authorities of our country. The objective is to design a Big Data system covering the four main requirements of large-scale data processing. 1 Support large writing workloads from various sources. 2 An elastic architecture, capable of withstanding the peak load times of work and adding or releasing resources as needed. 3 Support intensive analysis, to be able to admit large and diverse reading requests. 3 High availability to support errors in hardware and software.

Доступ адвоката до персональних даних свого клієнта у сфері надання медичної допомоги

Regulatory competition between the Law of Ukraine "On Personal Data Protection" and the Law of Ukraine "On Advocacy and Advocacy" has given rise to misinterpretations about a lawyer's access to information about the health of his client, his medical records and other confidential personal data. The analysis of law enforcement practice shows that the subjects of law have different understanding of the provisions of the Law of Ukraine "On Advocacy and Advocacy" and the Law of Ukraine "On Personal Data Protection" in terms of access to personal data of the client in the field of medical care. The essence of the problem is that today in the legal environment there are two opposing legal positions on the implementation of the right of a lawyer to access confidential information about the client. The lack of consensus on this issue, in our opinion, is due to the different understanding of law enforcement agencies of the concept of "restricted information", which is mentioned in the provisions of paragraph 3 of Part 1 of Art. 20 and Part 2 of Art. 24 of the Law of Ukraine "On Advocacy and Advocacy". According to Part 2 of Art. 21 of the Law of Ukraine "On Information" states that confidential information is about an individual, as well as information to which access is restricted by a natural or legal person, except for subjects of power. Clause 3, Part 4 of the same article of the Law of Ukraine “On Information” stipulates that information on medical care cannot be included in information with limited access. According to Art. 59 of the Constitution of Ukraine: “Everyone has the right to professional legal assistance. In cases provided by law, this assistance is provided free of charge. Everyone is free to choose a defender of their rights. " A lawyer is one of the key figures in jurisprudence, which stands for the protection of the rights, freedoms and legitimate interests of citizens. In order for a lawyer to be able to act confidently and effectively within the scope of his or her powers, the law confers on him or her rights that correspond to his or her functions. In accordance with Part 1 of Art. 20 of the Law of Ukraine "On Advocacy and Advocacy" a lawyer has the right to perform any actions not prohibited by law, rules of ethics and legal aid agreement, necessary for the proper implementation of the legal aid agreement, in particular: to make legal inquiries, including the receipt of copies of documents. However, as noted earlier, in practice, it is not uncommon for a lawyer to refuse to receive confidential information about his or her client. Judicial practice is dominated by decisions to bring to administrative responsibility for violations of legislation on personal data protection. The Constitutional Submission on the Conformity of the Constitution of Ukraine (constitutionality) with the provisions of Part 2 of Article 26 of the Law of Ukraine “On the National Police” [1] stipulates that the Law of Ukraine “On Personal Data Protection” is a framework law establishing general principles and requirements for personal data processing. While special laws should establish the rules of personal data processing in the relevant areas, taking into account the principles of the Law of Ukraine "On Personal Data Protection", taking into account the peculiarities of regulated relations, as well as provide appropriate mechanisms for protection against arbitrary restriction of privacy. It is obvious that one law cannot regulate the peculiarities of personal data processing for all spheres of activity of both public bodies and subjects of private law. The Law of Ukraine "On Advocacy and Advocacy" defines the legal basis for the organization and activities of advocacy and advocacy in Ukraine. Thus, all the features of the activity of lawyers, the legal limits of such, are established in a special law, the Law of Ukraine "On Advocacy and Advocacy". Therefore, in competition with the Law of Ukraine "On Advocacy and Advocacy" and the Law of Ukraine "On Personal Data Protection", in the activities of a lawyer when requesting copies of medical documents and information with limited access special rules are the Law of Ukraine "On Advocacy and Advocacy". The constitutional right of a person to professional legal assistance guarantees proper legal protection of his rights and interests in a manner prescribed by law. A lawyer has an unconditional right given to him by the client to receive information about his state of health, but we must not forget that the client's right to legal aid is not unlimited and ends where confidential information about another person begins.

Universal Rank-Order Transform to Extract Signals from Noisy Data

We introduce an ordinate method for noisy data analysis, based solely on rank information and thus insensitive to outliers. The method is nonparametric, objective, and the required data processing is parsimonious. Main ingredients are a rank-order data matrix and its transform to a stable form, which provide linear trends in excellent agreement with least squares regression, despite the loss of magnitude information. A group symmetry orthogonal decomposition of the 2D rank-order transform for iid (white) noise is further ordered by principal component analysis. This two-step procedure provides a noise "etalon" used to characterize arbitrary stationary stochastic processes. The method readily distinguishes both the Ornstein-Uhlenbeck process and chaos generated by the logistic map from white noise. Ranking within randomness differs fundamentally from that in deterministic chaos and signals, thus forming the basis for signal detection. To further illustrate the breadth of applications, we apply this ordinate method to the canonical nonlinear parameter estimation problem of two-species radioactive decay, outperforming special-purpose least square software. It is demonstrated that the method excels when extracting trends in heavy-tailed noise and, unlike the Thiele-Sen estimator, is not limited to linear regression. Lastly, a simple expression is given that yields a close approximation for signal extraction of an underlying generally nonlinear signal.

Smart media and application

Smart media and application

Consent for data processing under the General Data Protection Regulation: Could ‘dynamic consent’ be a useful tool for researchers?

The General Data Protection Regulation (GDPR) sets the bar high for consent for the processing of personal data. In the UK, researchers have been directed to rely on legal bases other than consent for processing personal data for research purposes. Informed consent, nonetheless, and despite certain shortcomings, holds a central position in ethical research practice, as well as at common law, and in a range of other legislation dealing with research involving humans. This paper evaluates the place of informed consent in research following the GDPR’s implementation, arguing that a fresh approach to consent — specifically the concept known as ‘dynamic consent’ — could provide a way for researchers to meet the new European regulatory requirements for data processing while adhering to the highest ethical standards for research conduct. It analyses dynamic consent according to specific GDPR requirements and reflects on practical examples that could inform future implementation of the approach, while remaining aware of the need for further empirical research.

An Efficient Match Search Approach Using Two-Dimensional Hash Function in Hardware-Based Dictionary Compression

The hardware-based dictionary compression is widely adopted for high speed requirement of real-time data processing. Hash function helps to manage large dictionary to improve compression ratio but is prone to collisions, so some phrases in match search result are not true matches. This paper presents a novel match search approach called dual chaining hash refining, which can improve the efficiency of match search. From the experimental results, our method showed obvious advantage in compression speed compared with other approach that utilizes single hash function described in the previous publications.

The unforeseen business and medical consequences of EHR data collection for a real-world data multiple myeloma registry.

e19528 Background: Record retrieval on behalf of a consenting patient within the context of real-world data is not well understood. As the need for real-world data continues to expand, methods for the efficient transfer of information across consenting parties will be critical to enable research collaborations. This need was highlighted through a partnership between COTA and the Multiple Myeloma Research Foundation (MMRF). As one component of the pilot study, COTA managed the record retrieval of consenting patients before going on to abstract the data for use in a registry. Methods: The pilot study identified 23 patients that consented to the release of medical records at 54 institutions across 20 states. COTA partnered with a retrieval vendor and employed its own outreach efforts for acquisition. Outreach and retrieval techniques were similar across COTA and the vendor, including targeted calls, delivery of IRB-approved consent materials, and on-site requests. The 30-day release parameter for a covered entity under 45 CFR 164.524(b)(2) of HIPAA’s Privacy Rule was used to evaluate the observed return rates. Results: A total of 56 medical records were requested, and 48 records were retrieved. The mean (±SD) retrieval time across all sites was 33 (±58) days. We found that 54% of records were released in &lt; = 30 days, and 32% of records &gt; 30 days. 14% of requested records were never released, despite a median of 19 outreach attempts (range 10 to 43). Cited issues for delay or non-release consisted of 22 institutions questioning the validity of the certified electronic patient signature, 6 requiring a physical signature, and 5 requiring their own authorization forms. In no cases did the records contain structured metadata, such as LOINC, MedDRA, and RxNorm. Conclusions: This pilot showed an unpredictable variance associated with the release of records in the context of real-world data. This variance contributed to barriers and delays in broader research efforts. The lack of accompanying metadata with released records resulted in additional required data processing. Future studies should be conducted to establish best practices in the release and retrieval of medical records used to support real-world data research.

Multiple Subspace Pursuit Joint Recovery Algorithm Based on Signal Correlation (SC-USP): Wireless Sensor Network Oriented to Rail Transit Environment

This paper is oriented to the wireless sensor network (WSN) application scenario in the railway transit environment, focusing on the large amount of signal transmission and data processing requirements required, making full use of the advantages of compressed sensing (CS), combined with existing mainstream compressed sensing technology such as Subspace Pursuit (SP). The technique of Multiple SP Joint Recovery Based on Signal Correlation (SC-USP) algorithm for a specific type of data signal is proposed. Finally, the simulation analysis shows the superiority of SC-USP in similar algorithms and application scenarios.

Problems and features of data structure modeling in distributed Web-oriented production systems

The goal is to develop a system model necessary for structuring the subject area, decomposing the functions performed by a web application; formalize the design methods, determine the general method of synthesizing Internet-oriented information systems. The main part of the development is the creation of a model, which components will be the objects of the domain, while attention should be paid to balance - the flexibility of the model and the development costs, taking into account the possibility of reuse. Too flexible components will have too many properties and will be difficult to customize adjust. On the contrary, models that implement too particular situation will be useless in terms of reuse, and, perhaps, the cost of its implementation will not make sense. For a more complete understanding of the functions and requirements for data processing, it is necessary to identify the main objects of the domain, information flows, logical connections, as well as rules and restrictions when working with data. The result of building a model should be a DBMS-independent data schema (metadata) corresponding to the internal data structure of the Web application, focused on the implementation of the required functions when interfacing with the user.

A Real-Time Tree Crown Detection Approach for Large-Scale Remote Sensing Images on FPGAs

The on-board real-time tree crown detection from high-resolution remote sensing images is beneficial for avoiding the delay between data acquisition and processing, reducing the quantity of data transmission from the satellite to the ground, monitoring the growing condition of individual trees, and discovering the damage of trees as early as possible, etc. Existing high performance platform based tree crown detection studies either focus on processing images in a small size or suffer from high power consumption or slow processing speed. In this paper, we propose the first FPGA-based real-time tree crown detection approach for large-scale satellite images. A pipelined-friendly and resource-economic tree crown detection algorithm (PF-TCD) is designed through reconstructing and modifying the workflow of the original algorithm into three computational kernels on FPGAs. Compared with the well-optimized software implementation of the original algorithm on an Intel 12-core CPU, our proposed PF-TCD obtains the speedup of 18.75 times for a satellite image with a size of 12,188 × 12,576 pixels without reducing the detection accuracy. The image processing time for the large-scale remote sensing image is only 0.33 s, which satisfies the requirements of the on-board real-time data processing on satellites.

Dataset retrieval system based on automation of data preparation with dataset description model

SummaryData preparation is the most effortful task in the process of statistical learning. Many studies related to data mining are performed without data preparation by assuming that qualified datasets are already prepared. It may hide useful patterns of data, which can result in poor performance and incorrect learning. Automation of data preparation can solve these problems. For automation of data preparation, a few issues should be considered, such as flexible expression of requirements according to the purpose of the learning model, accessibility to data sources, and performance degradation due to automation. In this paper, we propose a dataset description model that can express the requirements for data processing and dataset retrieval system based on automated data preparation. The proposed system makes it possible to provide good quality datasets for statistical learning applications using data preparation methods such as data acquisition, refinement, and organization. In the experiment, we demonstrate that the proposed system doesn't have performance loss as compared to the existing manual systems. Moreover, the quality of the datasets are also improved by using the proposed system.

Improving the sensitivity of FT-NMR spectroscopy by apodization weighted sampling

Apodization weighted acquisition is a simple approach to enhance the sensitivity of multidimensional NMR spectra by scaling the number of scans during acquisition of the indirect dimension(s). The signal content of the resulting spectra is identical to conventionally sampled data, yet the spectra show improved signal-to-noise ratios. There are no special requirements for data acquisition and processing: the time-domain data can be transformed with the same schemes used for conventionally recorded spectra, including Fourier transformation. The method is of general use in multidimensional liquid and solid state NMR experiments if the number of recorded transients per sampling point is bigger than the minimum required phase cycle of the pulse sequence.

Data Visualization Model Methods and Techniques

In order to meet the requirements of high-dimensional data processing in the information field, this paper aims to explore methods and techniques for visualizing general data resource clustering data. Through the visual mapping of dimensionality reduction and high-dimensional data, a visual learning model for visual influencing factors is established. The visual system model approach was tested using the IRIS dataset from the University of California Irvine database (UCL) database. The results show that the model can effectively analyze the data set, visualize the characteristics of IRIS data in real time, achieve the expected results, and point the way for other data visualization models.