Data Processing Requirements Research Articles

Modular DAS demodulation system based on ultra-weak fibre Bragg grating

Aiming at the requirements of low power consumption, miniaturization, and massive data processing, a modular distributed optical fibre acoustic sensing (DAS) demodulation system based on ultra-weak fibre Bragg grating (UW-FBG) is proposed. A highly integrated circuit and embedded system is used for the time-domain positioning and data acquisition of the gratings. Field programmable gate array (FPGA) demodulates the phase changes of each sensing section in real time. The demodulation outputs are uploaded through a Gigabit Ethernet connection. Through innovative software and hardware design, the demodulation system power consumption is less than 20 W, and the device volume is 250 mm× 180 mm× 45 mm, and the output data size is about 2.6% of the conventional methods. All the outputs are then tested on a piezoelectric transducer (PZT) platform. Results show that the background noise within 5 Hz–1280 Hz range is 2.16 pε/√(Hz), and the strain sensitivity reaches 484 pε/√(Hz)@200 Hz. The correlation coefficient of voltage and phase is 0.9991, which shows good linearity. Combined with a common commercial laptop to demodulate the micro-vibration signals, this system could provide a cost-effective scheme for acquiring acoustic-level micro-vibration signals and on-line field monitoring.

Universal Real-Time Adaptive Signal Compression for High-Frame-Rate Optoacoustic Tomography.

Optoacoustic tomography (OAT) has recently been advanced toward ultrafast volumetric imaging frame rates in the kilohertz range. As a result, excessive data processing and storage capacity requirements are increasingly being imposed on the imaging systems. OAT data commonly exhibit significant sparsity across the spatial, temporal or spectral domains, which facilitated the development of compressed sensing algorithms exploiting various sparse acquisition and under-sampling schemes to reduce data rates. However, performance of compressed sensing critically depends on a priori knowledge on the type of acquired data and/or imaged object, commonly resulting in lack of general applicability and unpredictable image quality. In this work, we report on a fast adaptive OAT data compression framework operating on fully sampled tomographic data. It is based on a wavelet packet transform that maximizes the data compression ratio according to the desired signal energy loss. A dedicated reconstruction method was further developed that efficiently renders images directly from the compressed data. Up to 1000x compression ratios were achieved while providing efficient control over the resulting image quality from arbitrary datasets exhibiting diverse spatial, temporal and spectral characteristics. Our approach enables faster and longer acquisitions and facilitates long-term storage of large OAT datasets.

CRAM-Seq: Accelerating RNA-Seq Abundance Quantification Using Computational RAM

RNA Sequence (RNA-Seq) abundance quantification is an important application in different fields of genomic studies, e.g., analysis offunctionally similar genes in a biological sample. This application depends on the availability of high volume of sequence data for high accuracy abundance estimation, which is made possible by next generation sequencing platforms. Large scale data processing requirements of this quantification application push conventional computing systems to their limits due to excessive data movement required between processing and memory elements. Processing-In-memory presents a viable solution to this drawback, through <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">in-situ</i> processing of the genomic data. In this paper, we present CRAM-Seq, an accelerator for RNA-Seq abundance quantification based on Computational RAM (CRAM) – an in-memory processing substrate capable of high degree of parallel processing with very low energy consumption. Through hardware/software co-design, we demonstrate that CRAM-Seq outperforms a commonly used state-of-the-art software abundance quantification algorithm, Kallisto – in terms of throughput and energy efficiency, while being highly scalable.

Load balance oriented data processing mechanism for bounded and unbounded data in smart cities

The co-existence of bounded data and unbounded data gives a great challenge for the traditional single and inflexible data processing in smart cities. The wide promotion of the internet of things (IoT) makes the data amount rapidly increase. This leads to the further raise of the requirement for data processing in smart cities, especially the demand for low latency and abundant data in real-time video services. To solve this problem, a load balance oriented data processing mechanism for bounded and unbounded data in smart cities is proposed. A smart city framework is introduced to explicit the role of data processing in smart cities. A load-balanced data processing mechanism is proposed. Based on the mathematical model for data processing in smart cities, the load-balanced data processing is abstracted into an optimization problem. Aiming to obtain the minimum load balance ratio (LBR), an LBR algorithm is presented. Through simulation and experiment, the superiority and feasibility of our work are validated via numerical simulation and prototype implementation, respectively.

Improvement of Monitoring Technology for Corrosive Pollution of Marine Environment under Cloud Computing Platform

In view of the increasingly serious problem of marine ecological environmental pollution, the traditional marine environmental corrosive pollution monitoring technology has poor monitoring accuracy and poor monitoring timeliness, and the improvement of the marine environmental corrosive pollution monitoring technology under the cloud computing platform is proposed. The research significance and corrosion influence factors of steel corrosion in the marine environment are described, and the research progress of corrosion mechanism in five different zones of the marine environment is reviewed. Cloud computing parallelizes the processing of corrosive pollution data in the marine environment through virtualization and distributed technology, which greatly improves the efficiency of the algorithm. This paper studies the existing cloud computing platform and ocean monitoring system architecture, uses the distributed architecture to design a cloud computing-oriented ocean monitoring system and meets the design requirements in data collection and data processing. The experimental results show that the precision of marine environmental corrosion pollution monitoring technology proposed in this paper is 96% on average, and the completion rate of monitoring images is 82% on average, which can effectively realize marine environmental corrosion pollution monitoring.

RUAP: Random rearrangement block matrix-based ultra-lightweight RFID authentication protocol for end-edge-cloud collaborative environment

Cloud computing provides powerful processing capabilities for large-scale intelligent Internet of things (IoT) terminals. However, the massive realtime data processing requirements challenge the existing cloud computing model. The edge server is closer to the data source. The end-edge-cloud collaboration offloads the cloud computing tasks to the edge environment, which solves the shortcomings of the cloud in resource storage, computing performance, and energy consumption. IoT terminals and sensors have caused security and privacy challenges due to resource constraints and exponential growth. As the key technology of IoT, Radio-Frequency Identification (RFID) authentication protocol tremendously strengthens privacy protection and improves IoT security. However, it inevitably increases system overhead while improving security, which is a major blow to low-cost RFID tags. The existing RFID authentication protocols are difficult to balance overhead and security. This paper designs an ultra-lightweight encryption function and proposes an RFID authentication scheme based on this function for the end-edge-cloud collaborative environment. The BAN logic proof and protocol verification tools AVISPA formally verify the protocol's security. We use VIVADO to implement the encryption function and tag's overhead on the FPGA platform. Performance evaluation indicates that the proposed protocol balances low computing costs and high-security requirements.

Visual Management of Sports Based on Intelligent Analysis of Big Data

After years of development, big data technology has become more mature and reliable in its application in various fields. With the explosive growth of data and the rise of data complexity, the processing and application of data have actually changed our daily life. The development of data in sports is very active in recent years, and the natural dependence of sports on data determines that the visualization of sports has a great creative space in data sports. Therefore, it is necessary to deeply study the visual management of sports under the background of big data technology. As we all know, in sports, competitive games occupy the vast majority, and the data generated by this is not limited to the score of the game. People are paying more and more attention to information related to sports, but these information are generally very complicated and huge. This is also an area with the widest audience, and the core of competitive sports is data. The final results and progress of most sports competitions are reflected through changes and comparisons of data. With the continuous development of big data information collection technology, the available sports data are more detailed, which also makes sports develop toward visualization. Therefore, with the help of the excellent ability of big data in screening data, this article proposes algorithms such as DTW and recurrent neural networks to reasonably and reliably analyze and process a large number of data generated in the process of sports and embeds an error analysis module in the designed model to ensure the accuracy requirements in data processing to a greater extent. The model designed in this article has been able to improve the accuracy by more than 86%. This will greatly facilitate our management and have a far-reaching impact on sports visualization.

Research on Graphic-Text Relationship in Film and Television Works Based on Big Data Model

With the rise of microblog, social network, mobile APP, location-based services, and other technologies, the global data has exploded. Big data provides a rich source for data mining and analysis, and the results mined from it will become more meaningful and even full of surprises. In this paper, through the problems in processing film and television data, the corresponding model is established to deal with the corresponding problems. According to the related technologies of big data processing and the filtering algorithm of the model, the processing strategies and methods are put forward. The middle also introduces the big data mining method. Finally, according to the different data characteristics and different data processing requirements of the intelligent analysis system of film and television data, comparing the data of film works and TV dramas, the experimental results show that, based on the big data model, the pictures and words in both film works and TV dramas are inseparable and indispensable. In the later research, we choose the appropriate data processing scheme to design and implement the data processing flow, and successfully apply the film and television big data processing strategy and method proposed in this paper to practice, which provides reliable data support for the content analysis of film and television works. The model-based filtering algorithm calculates the relationship between image and text, turns the point into the probability formula of node-to-meta path, and takes the approximate value to achieve efficient model learning.

Raising Consent Awareness With Gamification and Knowledge Graphs

Consent is one of GDPR’s lawful bases for data processing and specific requirements for it apply. Consent should be specific, unambiguous and most of all informed. However, an informed consent request does not guarantee having individuals who are aware of what it means to consent and the implications that follow. Consent is often given blindly now, in particular because of information overload from long privacy policies written in legal language and complex interface designs that cause consent fatigue on the users' side. This paper presents a knowledge graph-based user interface for consent solicitation, which uses gamification to raise the legal awareness and ease individual’s comprehension of consent. The knowledge graph models informed consent in a machine-readable format and provides a unified consent model to all entities involved in the data sharing process. The evaluation shows that with the help of gamification, the interface can raise individuals' average legal awareness to 92.86%.

Subsidence Detection for Urban Roads Using Mobile Laser Scanner Data

Pavement subsidence detection based on point cloud data acquired by mobile measurement systems is very challenging. First, the uncertainty and disorderly nature of object points data results in difficulties in point cloud comparison. Second, acquiring data with kinematic laser scanners introduces errors into systems during data acquisition, resulting in a reduction in data accuracy. Third, the high-precision measurement standard of pavement subsidence raises requirements for data processing. In this article, a data processing method is proposed to detect the subcentimeter-level subsidence of urban pavements using point cloud data comparisons in multiple time phases. The method mainly includes the following steps: First, the original data preprocessing is conducted, which includes point cloud matching and pavement point segmentation. Second, the interpolation of the pavement points into a regular grid is performed to solve the problem of point cloud comparison. Third, according to the high density of the pavement points and the performance of the pavement in the rough point cloud, using a Gaussian kernel convolution to smooth the pavement point cloud data, we aim to reduce the error in comparison. Finally, we determine the subsidence area by calculating the height difference and compare it with the threshold value. The experimental results show that the smoothing process can substantially improve the accuracy of the point cloud comparison results, effectively reducing the false detection rate and showing that subcentimeter-level pavement subsidence can be effectively detected.

Hyperspectral-Based Classification of Managed Permanent Grassland with Multilayer Perceptrons: Influence of Spectral Band Count and Spectral Regions on Model Performance

Detailed knowledge of botanical composition is a key factor for the sustainable and site-specific management of permanent grassland and facilitates an improvement in the performance and efficiency of livestock feeding. Spectral-based data acquisition combined with machine learning has the potential to classify species groups and plant parts in permanent grassland with high accuracy. However, a disadvantage of this method is the fact that hyperspectral sensors with a wide spectral range and fine spectral and high spatial resolution are costly and create large amounts of data. Therefore, the question arises as to whether these parameters are necessary for accurate grassland classification. Thus, the use of sensors with lower spectral and spatial resolution and correspondingly lower data processing requirements could be a conceivable approach. Therefore, we investigated the classification performance with reduced predictor sets formed by different approaches in permanent grassland. For pixel-based classification, a cross-validated mean accuracy of 86.1% was reached using a multilayer perceptron (MLP) including all 191 available predictors, i.e., spectral bands. Using only 48 high-performing predictors, an accuracy of 80% could still be achieved. In particular, the spectral regions of 954 nm to 956 nm, 684 nm to 744 nm and 442 nm to 444 nm contributed most to the classification performance. These results provide a promising basis for future data acquisition and the analysis of grassland vegetation.

A Workflow Demonstrator for Processing Catalysis Research Data

Abstract The UK Catalysis Hub (UKCH) is designing a virtual research environment to support data processing and analysis, the Catalysis Research Workbench (CRW). The development of this platform requires identifying the processing and analysis needs of the UKCH members and mapping them to potential solutions. This paper presents a proposal for a demonstrator to analyse the use of scientific workflows for large scale data processing. The demonstrator provides a concrete target to promote further discussion of the processing and analysis needs of the UKCH community. In this paper, we will discuss the main requirements for data processing elicited and the proposed adaptations that will be incorporated in the design of the CRW and how to integrate the proposed solutions with existing practices of the UKCH. The demonstrator has been used in discussion with researchers and in presentations to the UKCH community, generating increased interest and motivating further development.

Mengembangkan Kompetensi Guru Dalam Melaksanakan Evaluasi Pembelajaran Di Era Merdeka Belajar

One of the merdeka belajar (freedom of learning) concepts give teachers the freedom to make fun learning. Teachers must make plans, carry out, and evaluate their learning programs independently. Teacher becomes the determining factor for the success of the learing objectives. And to find out the success of the learningprogram we need to do an evaluation. Evaluation is an important activity, which is carried out to review the success of a program to determine a decision. Teachers must have competence to conduct evaluations. The objectives of this study include; 1) to understand the concept of evaluations. 2) to explain teacher competence in carring out evaluations. The method used in writing this journal is a literature review, where research data obtained from several literary theories are processed and analyzed and then described, so that the results are obtained that in the early stages, teachers must first understand the concept of evaluation, which emphasizes the definition, objectives , the function of the principle and type of evaluation, while the competence of teachers in evaluation activities includes competence in planning evaluation programs, designing evaluation programs, developing evaluation programs, implementing, and evaluating evaluation programs. The evaluation itself requires data processing, reporting and utilization of the evaluation results.

An efficient and secure aggregation encryption scheme in edge computing

Edge computing is a highly virtualized paradigm that can services the Internet of Things(IoT) devices more efficiently. It is a non-trivial extension of cloud computing, which can not only meet the big data processing requirements of cloud computing, but also collect and analyze distributed data. However, it inherits many security and privacy challenges of cloud computing, such as: authentication and access control. To address these problem, we proposed a new efficient privacy-preserving aggregation scheme for edge computing. Our scheme consists of two steps. First, we divided the data of the end users with the Simulated Annealing Module Partition (SAMP) algorithm. And then, the end sensors and edge nodes performed respectively differential aggregation mechanism with the Differential Aggregation Encryption (DAE) algorithm which can make noise interference and encryption algorithm with trusted authority (TA). Experiment results show that the DAE can preserve user privacy, and has significantly less computation and communication overhead than existing approaches.

GPU accelerated Cartesian GRAPPA reconstruction using CUDA

Background and ObjectiveGRAPPA (Generalized Auto-calibrating Partially Parallel Acquisition) is an advanced parallel MRI reconstruction method (pMRI) that enables under-sampled data acquisition with multiple receiver coils to reduce the MRI scan time and reconstructs artifact free image from the acquired under-sampled data. However, the reduction in MRI scan time comes at the expense of long reconstruction time. It is because the GRAPPA reconstruction time shows exponential growth with increasing number of receiver coils. Consequently, the conventional CPU platforms may not adhere to the requirements of fast data processing for MR image reconstruction. MethodsGraphics Processing Units (GPUs) have recently emerged as a viable commodity hardware to reduce the reconstruction time of pMRI methods. This paper presents a novel GPU based implementation of GRAPPA using custom built CUDA kernels, to meet the rising demands of fast MRI processing. The proposed framework exploits intrinsic parallelism in the calibration and synthesis phases of GRAPPA reconstruction process, aiming to achieve high speed MR image reconstruction for various GRAPPA configuration settings using different number of receiver coils, auto-calibration signals (ACS), sizes of GRAPPA kernel and acceleration factors. In-vivo experiments (using 8, 12 and 30 receiver coils) are performed to compare the performance of the proposed GPU accelerated GRAPPA with the CPU based GRAPPA extensions and GPU counterpart. ResultsThe results indicate that the proposed method achieves up to ≈47.8× , ≈17× and ≈3.8× speed up gains over multicore CPU (single thread), multicore CPU (8 thread) and Gadgetron (GPU based GRAPPA) respectively, without compromising the reconstruction accuracy. ConclusionsThe proposed method reduces the GRAPPA reconstruction time by employing the calibration phase (GRAPPA weights estimation) and synthesis phase (interpolation) on GPU. Our study shows that the proposed GPU based parallel framework for GRAPPA reconstruction provides a solution for high-speed image reconstruction while maintaining the quality of the reconstructed images.

Distributed collaboration and anti-interference optimization in edge computing for IoT

The edge computing (EC) systems for Internet of Things (IoT) can bring out low latency, high reliability, distributed intelligence, and network bandwidth savings to industrial real-time applications. However, limited computing and processing capabilities of edge devices remains to be difficult to meet complex data processing and artificial intelligence (AI) analysis requirements for diverse services. Besides, the scarcity of wireless spectrum resources in harsh industrial environment makes the interference between devices more serious. To address these challenges, this paper proposes an adaptive distributed collaborative anti-interference optimization scheme for IoT-edge system. Firstly, the EC system model is established, and the model of link failure probability is derived theoretically. Then, an Occupy-Interference Mitigation (O-IM) algorithm based on full-frequency multiplexing is proposed. The algorithm combines adaptive full-frequency multiplexing and interference mitigation to reduce the dependence of the reliable collaboration on bandwidth and signal-to-noise ratio (SNR). In addition, an anti-interference algorithm based on Interference Mitigation and inter-cellfrequency multiplexing Interference Avoidance (IM-IA) is proposed to balance bandwidth and SNR. This algorithm adopts interference cancellation scheme in the broadcasting phase, and adopts orthogonal frequency division in the collaboration phase. Extensive simulation results on Mininet platform verify that the proposals can obtain lower failure probability, and are less than traditional solutions in transmitting power, bandwidth, and SNR requirements. Moreover, the O-IM is suitable for low power scenarios, while the IM-IA is more suitable for high power case.

Comparison of Classical Methods and Mask R-CNN for Automatic Tree Detection and Mapping Using UAV Imagery

Detecting and mapping individual trees accurately and automatically from remote sensing images is of great significance for precision forest management. Many algorithms, including classical methods and deep learning techniques, have been developed and applied for tree crown detection from remote sensing images. However, few studies have evaluated the accuracy of different individual tree detection (ITD) algorithms and their data and processing requirements. This study explored the accuracy of ITD using local maxima (LM) algorithm, marker-controlled watershed segmentation (MCWS), and Mask Region-based Convolutional Neural Networks (Mask R-CNN) in a young plantation forest with different test images. Manually delineated tree crowns from UAV imagery were used for accuracy assessment of the three methods, followed by an evaluation of the data processing and application requirements for three methods to detect individual trees. Overall, Mask R-CNN can best use the information in multi-band input images for detecting individual trees. The results showed that the Mask R-CNN model with the multi-band combination produced higher accuracy than the model with a single-band image, and the RGB band combination achieved the highest accuracy for ITD (F1 score = 94.68%). Moreover, the Mask R-CNN models with multi-band images are capable of providing higher accuracies for ITD than the LM and MCWS algorithms. The LM algorithm and MCWS algorithm also achieved promising accuracies for ITD when the canopy height model (CHM) was used as the test image (F1 score = 87.86% for LM algorithm, F1 score = 85.92% for MCWS algorithm). The LM and MCWS algorithms are easy to use and lower computer computational requirements, but they are unable to identify tree species and are limited by algorithm parameters, which need to be adjusted for each classification. It is highlighted that the application of deep learning with its end-to-end-learning approach is very efficient and capable of deriving the information from multi-layer images, but an additional training set is needed for model training, robust computer resources are required, and a large number of accurate training samples are necessary. This study provides valuable information for forestry practitioners to select an optimal approach for detecting individual trees.

A community-based approach to image analysis of cells, tissues and tumors.

Emerging multiplexed imaging platforms provide an unprecedented view of an increasing number of molecular markers at subcellular resolution and the dynamic evolution of tumor cellular composition. As such, they are capable of elucidating cell-to-cell interactions within the tumor microenvironment that impact clinical outcome and therapeutic response. However, the rapid development of these platforms has far outpaced the computational methods for processing and analyzing the data they generate. While being technologically disparate, all imaging assays share many computational requirements for post-collection data processing. As such, our Image Analysis Working Group (IAWG), composed of researchers in the Cancer Systems Biology Consortium (CSBC) and the Physical Sciences - Oncology Network (PS-ON), convened a workshop on "Computational Challenges Shared by Diverse Imaging Platforms" to characterize these common issues and a follow-up hackathon to implement solutions for a selected subset of them. Here, we delineate these areas that reflect major axes of research within the field, including image registration, segmentation of cells and subcellular structures, and identification of cell types from their morphology. We further describe the logistical organization of these events, believing our lessons learned can aid others in uniting the imaging community around self-identified topics of mutual interest, in designing and implementing operational procedures to address those topics and in mitigating issues inherent in image analysis (e.g., sharing exemplar images of large datasets and disseminating baseline solutions to hackathon challenges through open-source code repositories).

Research on Construction Method of Wavelet Telemetry Data with Improved Threshold

Abstract In order to strengthen the applicability of data denoising algorithm, this thesis study the common telemetry data denoising algorithm based on the data of engine speed, flight space speed, cabin temperature and humidity, and establishes the evaluation model of error square sum and curve similarity to evaluate the denoising performance. Experiments show that the polynomial fitting has the greatest denoising error and slow convergence speed. The five-point cubic smoothing has the smallest overall denoising error, the median filtering algorithm can change the effect of smoothing effect by adjust it's moothing window, but ignores the authenticity of data. Therefore, the above three data denoising algorithms do not meet the requirements of telemetry data processing. In this thesis, an improved threshold function is proposed which effectively improves the data jump and excessive smoothing and reduce the denoising accuracy compared with the traditional thresholding function in order to makes the measured value closer to the true value. The algorithm is applied to the noise processing of four kinds of telemetry data, the results show that the denoising accuracy is improved significantly compared with the other three algorithms, which makes the measured value closer to the true value to reflect the changing trend of the original measurement data more truthfully, and the curve similarity is improved significantly, which are all above 80%.

The significance of product design in the circular economy: A sustainable approach to the design of data centre equipment as demonstrated via the CEDaCI design case study

Society depends on fast, uninterrupted 24/7 data transfer in almost every aspect of our lives. Data centres (DC) across the globe process billions of gigabytes of data every day. There are currently about 7 million data centres globally and the sector is expected to grow fivefold by 2030, inevitably increasing already high demand for millions of tonnes of resources, including critical raw materials (CRM). The infrastructure for recycling sectoral waste is lacking, wasting millions of tonnes of materials. Many virgin materials are located in conflict zones, which jeopardises the supply chain that the sector heavily relies on, consequently posing a danger of severe material shortages in the future. Present design thinking does not balance optimisation of the performance and operational energy requirements and designing for circularity. To date, server design has mainly been focused on energy efficiency. However, a Circular Economy (CE) entails consideration of a complete life cycle of the product, including embodied impacts. Because sectoral data processing and energy requirements continue to grow, sustainable design and innovation in line with the 2020 EU Circular Economy Action Plan are crucial in promoting best practices and managing the sector’s energy demand in the future. This paper describes a proposed sustainable server design developed as part of the CEDaCI (a Circular Economy for the Data Centre Industry) project and based on the design requirements outlined by the EU Circular Economy Action Plan 2020 together with other directives. It presents recommendations for good practice methods, futureproofing data centre equipment design for a Circular Economy and building a critical foundation for sustainable hardware design that can be developed in the future.