Reduce Data Size Research Articles

Power quality event classification using optimized Bayesian convolutional neural networks

Management of the electrical grid has an importance on the sustainability and reliability of the electrical energy supply. In the process, it is still crucial that power quality (PQ) is evaluated as part of any grid management master plan. This article provides a novel approach for classifying PQ disturbances such as voltage sag, swell, interruption and harmonics. In the proposed method, colorized continuous wavelet transform coefficients of the voltage signals are applied to convolutional neural networks as an image file. Thus, there is no need for extra feature selection and data size reduction steps as in conventional machine learning-based classifiers. Experiments were conducted on a dataset containing 1500 real-life disturbance signals measured from different locations in Turkey by Turkish Electricity Transmission Corporation. With the power of deep learning in image processing, the proposed method provides very high classification accuracy with a value of 99.8%. Comparisons with the other PQ disturbance classification methods, which are using traditional signal processing-based feature extraction and machine learning algorithm, prove that the proposed method has a simple methodology and overcomes the defects of these methods.

MinReduct: A new algorithm for computing the shortest reducts

Abstract This paper deals with the problem of computing the shortest reducts of a decision system. The shortest reducts are useful for attribute reduction in classification problems and data size reduction. Unfortunately, finding all the shortest reducts is an NP-hard problem. There are some algorithms reported in the literature to overcome the complexity of computing the shortest reducts. However, most of these algorithms relay on costly operations for candidate evaluation. In this paper, we propose a new algorithm for computing all the shortest reducts; based on binary cumulative operations over a pair-wise comparison matrix, and a fast candidate evaluation process. Binary cumulative operations save computation time by avoiding repetitive calculations. Furthermore, unlike other algorithms reported in the literature, our candidate evaluation process relays on low-cost operations which reduce the runtime in most cases. Our experiments over synthetic and real-world decision systems show that our proposal is faster than state of the art algorithms in most decision systems.

Information processing in Internet of Things using big data analytics

With innovation in persistent technologies, such as wearable sensor gadgets, sensor devices, and wireless ad-hoc communication networks connect everyday life things to the Internet, normally referred to as Internet of Things (IoT). IoT is observed as an active entity for design and development of smart and context awareness services and applications in the area of business, science and engineering discipline. These applications and services could vigorously respond to the surroundings transformation and users’ preference. Developing a scalable system for data analysis, processing and mining of enormous real world based datasets has turned into one of the demanding problems that faces both system research scholars and data management research scholars. Employing big data analytics with IoT technologies is one of the ways for handling the timely analyzing information (i.e., data, events) streams. In this paper, we propose an integrated approach that coalesce IoT systems with big data tools into a holistic platform for real-time and continuous data monitoring and processing. We propose Fog assisted IoT based Smart and real time healthcare information processing (SRHIP) system in which large amounts of data generated by IoT sensor devices are offloaded at Fog cloud form data analytics and processing with minimum delay. The processed data is then transferred to a centralized cloud system for further analysis and storage. In this work, we introduce a Fog-assisted model with big data environment for data analytic of real time data with remote monitoring and discuss our plan for evaluating its efficacy in terms of several performance metrics such as transmission cost, storage cost, accuracy, specificity, sensitivity and F-measure. The proposed SRHIP system needs less transmission cost of 40.10% in comparison to SPPDA, 100% fewer bytes are compromised in comparison to GCEDA. Our proposed system data size reduction of 60% reduction due to proposed compression scheme in comparison to other benchmark strategies that offer 40% of reduction.

Noise analysis using Tucker decomposition and PCA on spectral images

Given the improvement of Remote Sensing (RS) sensors, it has been possible to increase spatial and spectral resolution on many of them. Nevertheless, the amount of data to represent and post-process has become highly prohibitive. Therefore, the need to be able to process such huge data sets, and one of the possible ways to deal with problems is the use of compression methods, however, data loss happen if the need of data size reduction is a must. RS spectral imagery contain high quantities of redundant information along the spectral domain, thus, making possible to use compression methods effectively as for example, tensor decomposition algorithms. In Tucker decomposition (TKD), an interesting and strange phenomenon happens when spatial domain is maintained and spectral domain is reduced, as a preprocessing step of a semantic segmentation task. Under these conditions, it is possible to observe an improvement on Pixel Accuracy (PA) metric when it is compared with the same uncompressed spectral image. Therefore, this work presents a study on how noise affects the Tucker Decomposition compared with Principal Component Analysis (PCA) and its impact in semantic segmentation.

Recognition of partial discharge signals in impaired datasets using cumulative energy signatures

The problem of impaired data sets refers to data sets containing a vast majority of unwanted signals than signals of interest. With increased interest in partial discharge (PD) testing with arbitrary waveforms and transients, these kind of data sets are becoming more and more common. Traditional clustering techniques cannot be applied due to big differences in spatial densities of the existing clusters in the data set. This paper contributes a simple yet efficient technique to recognize PD signals from noise and other disturbances. The signal recognition features are based on two specific areas extracted from the cumulative energy signal (CE) of each recorded waveform. These areas weigh up the extent to which the recorded signals have a pulse-like shape. A third feature, defined as a shape factor, extracts additional metrics from the CE signal that serves the purpose of accounting for the factors affecting the computation of the proposed recognition features and threshold for data size reduction. These three CE-based features are used to create a graph from which a real PD can be spotted in large impaired data sets. The performance of this technique is tested using PD measurements from superimposed impulse tests on a 150 kV cable system.

Development and validation a nomogram for predicting the risk ofsevere COVID-19:Amulti-center study in Sichuan, China.

BackgroundSince December 2019, coronavirus disease 2019 (COVID-19) emerged in Wuhan and spread across the globe. The objective of this study is to build and validate a practical nomogram for estimating the risk of severe COVID-19.MethodsA cohort of 366 patients with laboratory-confirmed COVID-19 was used to develop a prediction model using data collected from 47 locations in Sichuan province from January 2020 to February 2020. The primary outcome was the development of severe COVID-19 during hospitalization. The least absolute shrinkage and selection operator (LASSO) regression model was used to reduce data size and select relevant features. Multivariable logistic regression analysis was applied to build a prediction model incorporating the selected features. The performance of the nomogram regarding the C-index, calibration, discrimination, and clinical usefulness was assessed. Internal validation was assessed by bootstrapping.ResultsThe median age of the cohort was 43 years. Severe patients were older than mild patients by a median of 6 years. Fever, cough, and dyspnea were more common in severe patients. The individualized prediction nomogram included seven predictors: body temperature at admission, cough, dyspnea, hypertension, cardiovascular disease, chronic liver disease, and chronic kidney disease. The model had good discrimination with an area under the curve of 0.862, C-index of 0.863 (95% confidence interval, 0.801–0.925), and good calibration. A high C-index value of 0.839 was reached in the interval validation. Decision curve analysis showed that the prediction nomogram was clinically useful.ConclusionWe established an early warning model incorporating clinical characteristics that could be quickly obtained on admission. This model can be used to help predict severe COVID-19 and identify patients at risk of developing severe disease.

Lensfree super-resolved imaging based on adaptive Wiener filter and guided phase retrieval algorithm

We present a data-efficient framework to achieve a super-resolved reconstruction in lensfree on-chip microscopy. Instead of using conventional 3D-stack holograms, our method accomplishes a 5 × reduction of data size without the loss of the imaging resolution, in which merely 30 raw holograms are recorded. The experiment of the resolution target demonstrates a half-pitch lateral resolution of 780 nm across a large field-of-view (FOV) of 28.6 mm2. For the view of H&E stained intestine pathology slide, our method shows an equivalent resolution but a 20-fold expansion of FOV in comparison with a 10 × commercial wide-field microscope. This work will be a useful tool for the miniaturization of whole slide pathological microscopy.

Analysis of compressive sensing and energy harvesting for wireless multimedia sensor networks

One of the main concerns of Wireless Multimedia Sensor Networks (WMSNs) is the huge data size causing the higher energy consumption in transmission. The high energy consumption is a critical problem for lifetime of network includes sensor nodes with limited battery. The data size reduction and Energy Harvesting (EH) methods are the promising solutions to improve the network lifetime. The main objective of this paper is to evaluate the impact of the different data size reduction methods, such as image compression and Compressive s Sensing (CS), and EH methods, such as vibration, thermal and indoor solar, on WMSNs lifetime in industrial environments. In addition, a novel Mixed Integer Programming (MIP) framework is proposed to maximize the network lifetime when EH, CS, and Error Control (EC) approaches are utilized together. Comparative performance results show that utilizing Binary Compressive Sensing (BCS) and Indoor Solar Harvester (ISH) extends industrial network lifetime significantly.

The concept of Laser Spot Thermography test rig with real-time data processing

The paper shows the concept of Laser Spot Thermography test rig with real time data processing. Laser Spot Thermography is a promising method for non-destructive testing applications especially, when fully non-contact procedure is required. Unfortunately, the procedure as of now is hard to apply, due to technical difficulties related with data processing. Laser thermography test procedure involves recording of multiple sequences (even hundreds), which have to be processed and evaluated as a whole. The industry requires quick and precise results, which is hard to achieve with the amount of output data from the currently known test procedure. The paper gives a proposal for improving the laser thermography test procedure through data parameterization (curve fitting) for infrared sequences. The data parameterization allows for reduction of output data size, in particular temporary data storage in RAM. The reduction of data size leads to faster real time processing.

Linear feature extraction from point cloud using color information

LiDAR is considered as an effective technology for digitizing the real scene at a very high-resolution and in a short time. However, the resolution of the LiDAR is not sufficient to identify and evaluate the façade surface features like edges and cracks. Generally, photographs provide a better interpretation of the linear characteristics. The complementary benefits of each allow exploring valuable spatial information with different surface detail levels. The paper introduces a flexible image-based approach for linear feature extraction from LiDAR point cloud. Initially, the algorithm converts the point clouds into a structured depth image to reduce the complexity and computation time. Using transformation matrix and camera calibration parameters, the visible point clouds are perceptively projected into color image space using co-linearity equations. The result depth channel is sampled with the interpolation process and added to the color channels to compute (RGBD) layers. The edges and linear features of the surface are initially extracted using the optical 2D imagery and subsequently, each pixel of the linear features can be projected directly into 3D space. Due to the various acquisition positions of the laser and color images, the issue of occlusion is resolved using the visibility algorithm. Applying the methodology, experimental results from the Treasury Monument of Jordan’s ancient Petra City, indicate that the developed approach provides adequate contour points for better interpretation and quantification of weathering processes and dangerous cracking. 3D Modeling these features can also reduce data size, facilitating surface inspection and analysis with simpler models.

AIS data analytics for adaptive rotating shift in vessel traffic service

PurposeConsidering the varied and dynamic workload of vessel traffic service (VTS) operators, design an adaptive rotating shift solution to prevent them from getting tired while ensuring continuous high-quality services and finally guarantee a benign maritime traffic environment.Design/methodology/approachThe problem of rotating shift in VTS and its influencing factors are analyzed first, then the framework of automatic identification system (AIS) data analytics is proposed, as well as the data model to extract spatial–temporal information. Besides, K-means-based anomaly detection method is adjusted to generate anomaly-free data, with which the traffic trend analysis and prediction are made. Based on this knowledge, strategies and methods for adaptive rotating shift design are worked out.FindingsIn VTS, vessel number and speed are identified as two most crucial factors influencing operators' workload. Based on the two factors, the proposed data model is verified to be effective on reducing data size and improving data processing efficiency. Besides, the K-means-based anomaly detection method could provide stable results, and the work shift pattern planning algorithm could efficiently generate acceptable solutions based on maritime traffic information.Originality/valueThis is a pioneer work on utilizing maritime traffic data to facilitate the operation management in VTS, which provides a new direction to improve their daily management. Besides, a systematic data-driven solution for adaptive rotating shift is proposed, including knowledge discovery method and decision-making algorithm for adaptive rotating shift design. The technical framework is flexible and can be extended for managing other activities in VTS or adapted in diverse fields.

Development of a system for efficient content-based retrieval to analyze large volumes of climate data

Analyses of large ensemble data on future climate are significantly useful for the probabilistic future projection of climate change in various interdisciplinary fields. However, the data volume of the Database for Policy Decision making for Future climate change or d4PDF, which is a mega-ensemble dataset, exceeds ∼ 3 PB, which is too large to download to local computers. To allow users for retrieve and downloading necessary data, we developed a user-friendly system called “System for Efficient content-based retrieval to Analyze Large volume climate data” (SEAL) under the Social Implementation Program on Climate Change Adaptation Technology (SI-CAT). Conventional web-based retrieval systems allow retrievals using metadata associated with a data file itself. In contrast, SEAL allows the users to retrieve the necessary data by using metadata associated with contents, such as physical values, of a data file. We confirmed that SEAL can reduce data sizes and total time required for obtaining necessary data to less than 0.5% and 1%, respectively, compared to conventional web-based retrieval systems.

Cloud-Based Vehicle Tracking System

The need for vehicle tracking system in real time is growth continues due to increase the cases of theft. This type of system in real time needs to transmit large data with huge number of HTTP request to the server to keep tracking and monitoring in real time, thus causes spend extremely high cost every month for transportation the information on tracking vehicles to server therefor the needs for reducing the number of transportation and data size that transmits in each HTTP request to save expenses. This paper designed and implement an integrated vehicle tracking system in real time to track vehicle anywhere and anytime. This system is divided into two parts: vehicle tracking part and monitoring part. Tracking part is represented by installation the electronic devices in the vehicle using modern Global Positioning System (GPS), microcontroller Arduino UNO R3 and SIM800L GSM/GPRS modem. GPS is determined location of the vehicle via received coordinates from satellites such as latitude and latitude with accuracy ranging approximately 2.5 meters; the coordinates faked to add a type of protection to information on vehicles without effecting on characterizing real time tracking before sending it via a General Packet Radio service (GPRS). The monitoring part is in the cloud and will receive the coordinates and displays it on a map in a web page. The main contribution of this system is it reduced data size that sent from in-vehicle device via selected only necessary data for tracking vehicle from NEMA sentences of GPS and reduced number of HTTP request that sent to remote server via constrain the transmission of information with the movement of vehicles, since when vehicle moved the coordinates each 10s and did not send anything when the vehicle stopped thus will reduce the cost of expenses every month. This system can be utilized to track and monitoring the vehicles of large universities, companies, organization and also can be used in army vehicles and police vehicles.
 
 

SnowPac: a multiscale cubic B-spline wavelet compressor for astronomical images

ABSTRACT As more advanced and complex survey telescopes are developed, the size and scale of data being captured grows at increasing rates. Across various domains, data compression through wavelets has enabled the reduction of data size and increase in computation efficiency. In this paper, we provide qualitative and quantitative tests of a new wavelet-based image compression method compared against the current standard for astronomical images. The analysis is improved by making use of state-of-the-art object detection systems to accurately measure the impact of the compression. We find that a combination of lossy wavelet-based methods, efficient quantization, and lossless dictionary compressors can preserve up to 98 per cent of astronomical objects at a 10:1 compression ratio. This significant reduction in file size also preserves astronomical object properties better than existing methods. These methods help further reduce future workloads for image-heavy processing pipelines.

A Data Size Reduction Approach Applicable in Process Control System of Oil and Gas Plants

In oil and gas plants, the cost of devices applicable for supervising and controlling systems directly depends on the transmission and storage systems, which are related to the data size of process variables. In this paper, process variables frequency-domain and statistical analysis results have been studied to infer if there exists any possibility to reduce data size of the process variables without loss of any necessary information. Although automatic control is not applicable in a shutdown condition, for generalization of the obtained results, unscheduled shutdown data has also been analyzed and studied. The main goal of this paper is to develop an applicable algorithm for oil and gas plants to decrease the data size in controlling and monitoring systems, based on well-known and powerful mathematical techniques. The results show that it is possible to reduce the size of data dramatically (more than 99% for controlling, and more than 55% for monitoring purposes in comparison with existing methods), without loss of vital information and performance quality.

EEG Compression Using Motion Compensated Temporal Filtering and Wavelet Based Subband Coding

Electroencephalography (EEG) signals are commonly used in medical applications for prevention, diagnosis, and detection of neurological diseases. These EEG signals have also been used in designing brain computer interfaces for assistive technologies. For densely placed electrodes and long EEG recordings, a large amount of data needs to be stored, preferably in compressed form. This EEG signal compression is particularly required in an out-of-the-lab environment so that these signals are efficiently transmitted over wired/wireless communication channels. To this end, we propose a novel compression scheme for EEG signals, which exploits the intra-channel redundancy using motion compensated temporal filtering (MCTF) and discrete wavelet transform (DWT) based sub-band coding. In the pre-processing stage, multi-frame data is constructed such that each group of picture (GOP) contains information from a single channel of EEG data. This helps in removing the intra-channel redundancy. We apply MCTF on each GOP to exploit temporal redundancy, following which the DWT is applied on temporally decomposed frames to exploit spatial redundancy. Each spatio-temporal decomposed frame is assigned a bit budget for minimum distortion. For this purpose, we assign more bit budget to temporally decomposed low pass frames as compared to high pass frames. Spatio-temporal frames are then encoded at the assigned bit rate by using set partitioning in hierarchical tree (SPIHT) algorithm to create the bit stream. Our experimental results showed 4.5% and 2.4% reduction in distortion at the same data rate for BCI-3 and BCI-4 datasets, respectively. These results improve upon the reduction in data size achieved using state-of-the-art compression methods such as SPIHT and SPIHT with independent component analysis.

Less is more: On-board lossy compression of accelerometer data increases biologging capacity.

GPS‐tracking devices have been used in combination with a wide range of additional sensors to study animal behaviour, physiology and interaction with their environment. Tri‐axial accelerometers allow researchers to remotely infer the behaviour of individuals, at all places and times. Collection of accelerometer data is relatively cheap in terms of energy usage, but the amount of raw data collected generally requires much storage space and is particularly demanding in terms of energy needed for data transmission.Here, we propose compressing the raw accelerometer (ACC) data into summary statistics within the tracking device (before transmission) to reduce data size, as a means to overcome limitations in storage and energy capacity.We explored this type of lossy data compression in the accelerometer data of tagged Bewick's swans Cygnus columbianus bewickii collected in spring 2017. Using software settings in which bouts of 2 s of both raw ACC data and summary statistics were collected in parallel but with different bout intervals to keep total data size comparable, we created the opportunity for a direct comparison of time budgets derived by the two data collection methods.We found that the data compression in our case yielded a six times reduction in data size per bout, and concurrent, similar decreases in storage and energy use of the device. We show that with the same accuracy of the behavioural classification, the freed memory and energy of the device can be used to increase the monitoring effort, resulting in a more detailed representation of the individuals’ time budget. Rare and/or short behaviours, such as daily roost flights, were picked up significantly more when collecting summary statistics instead of raw ACC data (but note differences in sampling rate). Such level of detail can be of essential importance, for instance to make a reliable estimate of the energy budgets of individuals.In conclusion, we argue that this type of lossy data compression can be a well‐considered choice in study situations where limitations in energy and storage space of the device pose a problem. Ultimately, these developments can allow for long‐term and nearly continuous remote monitoring of the behaviour of free‐ranging animals.

Ed-PUF: Event-Driven Physical Unclonable Function for Camera Authentication in Reactive Monitoring System

As surveillance footage plays an increasingly significant role in law enforcement, it is imperative to ensure the integrity of recorded video data and the authenticity of its originator, and instill situation awareness into these monitoring systems with a fidelity record of the incidents. Unfortunately, existing frame-based networked surveillance systems could only partially fulfill these requirements. The emerging Dynamic Vision Sensor (DVS) sheds new light on solving this problem with its completely different sensor design, i.e., DVS responds only to temporal intensity change and records only sparse asynchronous address-events with precise timing information. Motivated by the reduced data size of activities and the prevention of privacy intrusion of subjects under surveillance as well as other appealing attributes, this work introduces the first event-driven physical unclonable function (Ed-PUF) system to fill the forensic gap of simultaneously authenticating the event data integrity and source camera identity for reactive monitoring by DVS camera. New DVS sensor architecture is proposed with negligible modifications made to the original DVS pixel. The Ed-PUF response bit can only be triggered by and uniquely dependent on the asynchronous addressed event without being interfered by the simultaneous firing of other address events. Address event streams are securely transmitted with an event package tag created by a keyed hash-based message authentication code with the key being the Ed-PUF response. A secure protocol to authenticate the identity of DVS camera and the integrity of address events transmitted through cellular network is also proposed. A camera lock is embedded to protect against severing and splicing the inter-chip connectivity within the camera for raw PUF responses. The proposed system is evaluated using raw PUF data obtained by post-layout Monte Carlo simulation in UMC 180nm technology and real event stream captured by a DVS camera. The proposed Ed-PUF has been demonstrated to have excellent uniqueness, randomness and reliability. Collision test is also conducted to show that the quality of DVS imaging is not compromised. Besides keeping the hardware/power/timing overheads low, the proposed scheme is also analyzed to be resilient against multiple attack scenarios.

New Data Structure for Many-to-Many Relations to Reduce Data Size, Recording Time, and Search Time

New Data Structure for Many-to-Many Relations to Reduce Data Size, Recording Time, and Search Time

No Soldiers Left Behind: An IoT-Based Low-Power Military Mobile Health System Design

There has been an increasing prevalence of ad-hoc networks for various purposes and applications. These include Low Power Wide Area Networks (LPWAN) and Wireless Body Area Networks (WBAN) which have emerging applications in health monitoring as well as user location tracking in emergency settings. Further applications can include real-time actuation of IoT equipment, and activation of emergency alarms through the inference of a user's situation using sensors and personal devices through a LPWAN. This has potential benefits for military networks and applications regarding the health of soldiers and field personnel during a mission. Due to the wireless nature of ad-hoc network devices, it is crucial to conserve battery power for sensors and equipment which transmit data to a central server. An inference system can be applied to devices to reduce data size for transfer and subsequently reduce battery consumption, however this could result in compromising accuracy. This paper presents a framework for secure automated messaging and data fusion as a solution to address the challenges of requiring data size reduction whilst maintaining a satisfactory accuracy rate. A Multilayer Inference System (MIS) was used to conserve the battery power of devices such as wearables and sensor devices. The results for this system showed a data reduction of 97.9% whilst maintaining satisfactory accuracy against existing single layer inference methods. Authentication accuracy can be further enhanced with additional biometrics and health data information.