Reduce Data Size Research Articles

Minimum data sampling requirements for accurate detection of terrain-induced gait alterations change with mobile sensor position

Human gait is a key biomarker for health, independence and quality of life. Advances in wearable inertial sensor technologies have paved the way for out-of-the-lab human gait analysis, which is important for the assessment of mobility and balance in natural environments and has applications in multiple fields from healthcare to urban planning. Automatic recognition of the environment where walking takes place is a prerequisite for successful characterisation of terrain-induced gait alterations. A key question which remains unexplored in the field is how minimum data requirements for high terrain classification accuracy change depending on the sensor placement on the body. To address this question, we evaluate the changes in performance of five canonical machine learning classifiers by varying several data sampling parameters including sampling rate, segment length, and sensor configuration. Our analysis on two independent datasets clearly demonstrate that a single inertial measurement unit is sufficient to recognise terrain-induced gait alterations, accuracy and minimum data requirements vary with the device position on the body, and choosing correct data sampling parameters for each position can improve classification accuracy up to 40% or reduce data size by 16 times. Our findings highlight the need for adaptive data collection and processing algorithms for resource-efficient computing on mobile devices.

HOI-V: One-stage human-object interaction detection based on multi-feature fusion in videos

Effective detection of Human-Object Interaction (HOI) is important for machine understanding of real-world scenarios. Nowadays, image-based HOI detection has been abundantly investigated, and recent one-stage methods strike a balance between accuracy and efficiency. However, it is difficult to predict temporal-aware interaction actions from static images since limited temporal context information is introduced. Meanwhile, due to the lack of early large-scale video HOI datasets and the high computational cost of spatial-temporal HOI model training, recent exploratory studies mostly follow a two-stage paradigm, but independent object detection and interaction recognition still suffer from computational redundancy and independent optimization. Therefore, inspired by the one-stage interaction point detection framework, a one-stage spatial-temporal HOI detection baseline is proposed in this paper, in which the short-term local motion features and long-term temporal context features are obtained by the proposed temporal differential excitation module (TDEM) and DLA-TSM backbone. Complementary visual features between multiple clips are then extracted by multi-feature fusion and fed into the parallel detection branches. Finally, a video dataset containing only actions with reduced data size (HOI-V) is constructed to motivate further research on end-to-end video HOI detection. Extensive experiments are also conducted to verify the validity of our proposed baseline.

Eigenbin compression for reducing photon-counting CT data size.

Photon-counting CT (PCCT) systems acquire multiple spectral measurements at high spatial resolution, providing numerous image quality benefits while also increasing the amount of data that must be transferred through the gantry slip ring. This study proposes a lossy method to compress photon-counting CT data using eigenvector analysis, with the goal of providing image quality sufficient for applications that require a rapid initial reconstruction, such as to confirm anatomical coverage, scan quality, and to support automated advanced applications. The eigenbin compression method was experimentally evaluated on a clinical silicon PCCT prototype system. The proposed eigenbin method performs principal component analysis (PCA) on a set of PCCT calibration measurements. PCA finds the orthogonal axes or eigenvectors, which capture the maximum variance in the N dimensional photon-count data space, where N is the number of acquired energy bins. To reduce the dimensionality of the PCCT data, the data are linearly transformed into a lower dimensional space spanned by the M<N eigenvectors with highest eigenvalues (i.e., the vectors that account for most of the information in the data). Only M coefficients are then transferred per measurement, which we term eigenbin values. After transmission, the original N energy-bin measurements are estimated as a linear combination of the M eigenvectors. Two versions of the eigenbin method were investigated: pixel-specific and pixel-general. The pixel-specific eigenbin method determines eigenvectors for each individual detector pixel, while the more practically realizable pixel-general eigenbin method finds one set of eigenvectors for the entire detector array. The eigenbin method was experimentally evaluated by scanning a 20cm diameter Gammex Multienergy phantom with different material inserts on a clinical silicon-based PCCT prototype. The method was evaluated with the number of eigenbins varied between two and four. In each case, the eigenbins were used to estimate the original 8-bin data, after which material decomposition was performed. The mean, standard deviation, and contrast-to-noise ratio (CNR) of values in the reconstructed basis and virtual monoenergetic images (VMI) were compared for the original 8-bin data and for the eigenbin data. The pixel-specific eigenbin method reduced photon-counting CT data size by a factor of four with<5% change in mean values and a small noise penalty (mean change in noise of<12%, maximum change in noise of 20% for basis images). The pixel-general eigenbin compression method reduced data size by a factor of 2.67 with<5% change in mean values and a less than 10% noise penalty in the basis images (average noise penalty≤5%). The noise penalty and errors were less for the VMIs than for the basis images, resulting in<5% change in CNR in the VMIs. The eigenbin compression method reduced photon-counting CT data size by a factor of two to four with less than 5% change in mean values, noise penalty of less than 10%-20%, and change in CNR ranging from 15% decrease to 24% increase. Eigenbin compression reduces the data transfer time and storage space of photon-counting CT data for applications that require rapid initial reconstructions.

An efficient data compression and storage technique with key management authentication in cloud space

Cloud computing is one of the promising technologies that offers cost-effective choices for processing and storing the huge volumes of data. In today’s world, data is the most important asset that one can have but it needs to be handled and protected properly. Portability of data can be increased by reducing the size of the data to be stored because of the limited storage space. As a result, data compression has arisen significantly. Data compression is a useful technique for reducing data size and increasing the effectiveness of data transit and storage. Data compression reduces the size of a data file while using lossy or lossless compression. One of the newest techniques for data compression is data duplication, which can reduce the amount of data saved while removing unnecessary data and maintaining an exact copy of the data. This analysis presents an Efficient data compression and storage technique with key management authentication in cloud space. This approach uses Regressive probabilistic key encryption (RPKE) to encrypt the cloud data and Lempel-Ziv-77-Huffman coding (LZ77-HM) is used to compress the huge amounts of cloud data. The Performance of presented approach is evaluated in terms of compression ratio and compression rate.

Machine learning based battery pack health prediction using real-world data

The complex operational conditions in real-world electric vehicles (EVs) contribute to the complexity of managing and maintaining battery packs. Adding to these challenges is the intricate task of modeling the inconsistent coupling among individual cells within these packs. This study addresses the ongoing challenges in modeling lithium-ion battery (LIB) cells within packs and estimating their state of health (SOH) for practical applications. This research proposed a PCA-CNN-Transformer method to model and predict the SOH model of real-world EV. Three main contributions are presented: a novel approach to defining an attenuation SOH model based on delivered energy, a methodology utilizing Principal Component Analysis (PCA) for cell modeling, and an SOH estimation model employing CNN-Transformer architecture. To address both pack and cell-level modeling, a hierarchical feature extraction approach is proposed. The health features extracted from both levels are assessed using grey relational analysis, showing a strong correlation with LIB SOH, exceeding 0.70. The proposed cell modeling method significantly reduces data size by 96%, enhancing computational efficiency. Furthermore, the integration of 1D-CNN in the SOH estimation model overcomes the limitations of the attention mechanism, achieving a MAE with 0.0406 and r-square of 0.9327, improved the original transformer network performance by 10.95%. This study also examines and discusses the performance of the informer and transformer models, elaborating why the informer model underperformed in this dataset.

FPGA implementation of automatic seizure detection in EEG signals using machine learning algorithm

This work presents a novel approach that harnesses the capabilities of field programmable gate arrays (FPGA) to enable instantaneous monitoring of electroencephalography (EEG) signals for the detection and prediction of seizure patterns. The integration of FPGA technology with the perceptron learning algorithm holds promises for enhancing real-time healthcare monitoring systems and contributing to improved patient care and safety. The proposed work addresses the challenges presented at predicting seizures from EEG data. The first challenge is to handle massive EEG data that poses memory issues. To tackle this, the research employs cellular automata (Rule 90 and Rule 150) to reduce data size by 85.71%, making it more manageable. The second is to implement a linearly classified perceptron algorithm using FPGA to predict and detect seizures from real-time EEG data. The third is clock synchronization between stored data and real EEG data for comparison purposes. The proposed system is continuously compared to real-time EEG signals using a single perceptron neural network, and alert signals are generated based on preactivation values, with a continuous alert issued when the value reaches 3/4th sample match. These early alerts empower individuals to take preventive actions. The model is implemented using an FPGA Zynq-7000 series, which consumes 270mW of power. The design utilizes 118 Lookup Tables (LUTs).

Permafrost Distribution in the Southern Carpathians, Romania, Derived From Machine Learning Modeling

ABSTRACTComputer modeling of sporadic and isolated patches of mountain permafrost distribution is difficult to implement without overestimating it. The main challenge is to determine the very areas where the criteria for permafrost maintenance are met. This paper aims to modeling the permafrost distribution in the Southern Carpathians (SC), a typical marginal periglacial mountain range. For this purpose, a collection of 883 bottom temperature of late winter snow cover (BTS) points was used as a proxy for permafrost presence or absence in order to train several machine learning models. The performances of each model were evaluated with AUC with varying between 0.99 for Maxent and 0.74 for K‐nearest neighbors and most models (five) exhibiting values between 0.82 and 0.86. Other tests such as confusion matrices, sensitivity analyses, data shuffling, and data size reduction tests indicated that Maxent, AdaBoost, and support vector machine offered the best results while logistic regression, neural network, and gradient boosting exhibited rather poor permafrost distributions. The final ensemble median model indicated a total permafrost area of 19.2 km2 occupying 1%–9% of the alpine area of the studied massifs. NDVI proved crucial for permafrost prediction because it allows delimiting the debris surfaces where permafrost is probable.

Hierarchical Autoencoder-Based Lossy Compression for Large-Scale High-Resolution Scientific Data

Lossy compression has become essential an important technique to reduce data size in many domains. This type of compression is especially valuable for large-scale scientific data, whose size ranges up to several petabytes. Although Autoencoder-based models have been successfully leveraged to compress images and videos, such neural networks have not widely gained attention in the scientific data domain. Our work presents a neural network that not only significantly compresses large-scale scientific data, but also maintains high reconstruction quality. The proposed model is tested with scientific benchmark data available publicly and applied to a large-scale high-resolution climate modeling data set. Our model achieves a compression ratio of 140 on several benchmark data sets without compromising the reconstruction quality. 2D simulation data from the High-Resolution Community Earth System Model (CESM) Version 1.3 over 500 years are also being compressed with a compression ratio of 200 while the reconstruction error is negligible for scientific analysis.

Efficient Data Management in Agricultural IoT: Compression, Security, and MQTT Protocol Analysis.

The integration of Internet of Things (IoT) technology into agriculture has revolutionized farming practices by using connected devices and sensors to optimize processes and facilitate sustainable execution. Because most IoT devices have limited resources, the vital requirement to efficiently manage data traffic while ensuring data security in agricultural IoT solutions creates several challenges. Therefore, it is important to study the data amount that IoT protocols generate for resource-constrained devices, as it has a direct impact on the device performance and overall usability of the IoT solution. In this paper, we present a comprehensive study that focuses on optimizing data transmission in agricultural IoT solutions with the use of compression algorithms and secure technologies. Through experimentation and analysis, we evaluate different approaches to minimize data traffic while protecting sensitive agricultural data. Our results highlight the effectiveness of compression algorithms, especially Huffman coding, in reducing data size and optimizing resource usage. In addition, the integration of encryption techniques, such as AES, provides the security of the transmitted data without incurring significant overhead. By assessing different communication scenarios, we identify the most efficient approach, a combination of Huffman encoding and AES encryption, to strike a balance between data security and transmission efficiency.

Color-to-Gray Conversion Method Based on Chroma Difference

Most of the images encountered in daily life are color images, yet grayscale remains prevalent in many fields due to its reduced data size and simplified operations. When reducing the dimensions of a three-channel color image to a single-channel grayscale, a portion of the original color information is inevitably lost. To yield high-quality grayscale images, this study introduces a color-to-gray conversion method based on chroma difference. This method defines a novel color distance metric for color-to-grayscale conversion, incorporating pixel chromaticity differences alongside brightness variations. The discrepancy in gray pixel values in the output grayscale image effectively mirrors the overall differences among input color image pixels. Optimization is conducted using the conjugate gradient method, ensuring appropriate reflection of luminance information from the input image and chromaticity data from the original color image within the grayscale rendering. Experimental validation confirms the efficacy of this approach. However, as the method accounts for all pixel pairs, it occasionally considers unnecessary pairs, leading to potential distortion in color differences between pixels and consequent inadequacies in chromaticity variation. To address this issue, a weighting factor is introduced, prioritizing color combinations with similar color differences. Experimental findings demonstrate that grayscale images produced using the proposed method outperform those generated by alternative approaches in preserving color differentials and retaining detailed features of the original image. The resulting output exhibits clear and natural outlines, as supported by both subjective and objective assessments.

Circuit-Level Realization of an Adaptive Trending Prediction Algorithm for Lossless ECG Compression

This paper introduces an advanced approach to the hardware implementation of lossless compression for electrocardiogram (ECG) signals, combining Content Adaptive Golomb Rice (CAGR) coding with Adaptive Trending Prediction (ATP). The CAGR technique dynamically adjusts its parameters based on local signal characteristics, effectively optimizing compression efficiency. Simultaneously, Adaptive Trending Prediction analyzes and predicts signal trends to further minimize redundancy before encoding, enhancing overall compression ratios. The hardware architecture presented in this study is meticulously designed to leverage these techniques, ensuring efficient utilization of resources while maintaining real time performance suitable for medical applications. Design considerations include algorithmic complexity, throughput, and resource utilization metrics tailored to meet the stringent requirements of medical monitoring systems. Experimental evaluations validate the efficacy of the proposed method, demonstrating substantial reductions in data size while preserving clinical fidelity crucial for accurate diagnosis and monitoring. The findings underscore the feasibility and practicality of integrating advanced compression algorithms into dedicated hardware platforms, thereby facilitating more efficient data transmission and storage in medical environments.

Beyond JSON: Evaluating Serialization Formats for Space-Efficient Communication

Distributed systems rely on efficient inter-service communication, heavily impacted by data transmission costs. This study investigates alternative serialization formats, like Avro and MessagePack, to reduce data size compared to the common JSON format. We utilize a custom model to comprehensively assess the space efficiency of serialization formats across various data types. Our findings demonstrate that adopting alternative formats achieves a median reduction in serialized data exceeding 30 %. Notably, Avro exhibits exceptional efficiency, leading to reductions exceeding 83 % in specific scenarios. These insights empower developers to select optimal formats, potentially leading to significant improvements in data transfer speed, reduced bandwidth consumption, and enhanced scalability for handling larger data volumes within distributed systems.

A Novel Approach of Cloud Computing Network for Authentication and Security Enhancement of IoT Enabled Cancer Forecasting System

Recently, a variety of approaches have been employed to address a broad spectrum of real-world issues; these methodology cover a variety of areas, including healthcare systems. Previous researchers concentrated on health-care monitoring software. It had several shortcomings, such as poor health-care data storage, time, expense, and processing complexity. This paper proposes a unique IoT-enabled and secured clinical monitoring paradigm to address these issues. Initially, implant several sensors to gather information on vital indicators like body temperature fluctuation. Phone numbers, marital status, heart rate deviation, residence, name, age, and blood pressure are among the patient's health information. The IoT medical sensor dataset is used in this investigation. During pre-processing, extra unnecessary attributes are removed, resulting in data size reduction and normalization. A convolutional neural network supporting the classification of cancer sickness that is based on the improved teaching-learning optimization (CNN-ITLO) method. The CNN-ITLO model ascertains whether or not the patient is cancer-prone based on the sensor input. The management of the hospital receives the gathered data after which it is analyzed. Lastly, the homomorphic encryption approach is used to encrypt and store the patient's data on the cloud. The Java platform will be used to carry out the recommended work. According to the experiment results, the suggested method performed better than current cutting-edge practices.

Transition role of entangled data in quantum machine learning

Entanglement serves as the resource to empower quantum computing. Recent progress has highlighted its positive impact on learning quantum dynamics, wherein the integration of entanglement into quantum operations or measurements of quantum machine learning (QML) models leads to substantial reductions in training data size, surpassing a specified prediction error threshold. However, an analytical understanding of how the entanglement degree in data affects model performance remains elusive. In this study, we address this knowledge gap by establishing a quantum no-free-lunch (NFL) theorem for learning quantum dynamics using entangled data. Contrary to previous findings, we prove that the impact of entangled data on prediction error exhibits a dual effect, depending on the number of permitted measurements. With a sufficient number of measurements, increasing the entanglement of training data consistently reduces the prediction error or decreases the required size of the training data to achieve the same prediction error. Conversely, when few measurements are allowed, employing highly entangled data could lead to an increased prediction error. The achieved results provide critical guidance for designing advanced QML protocols, especially for those tailored for execution on early-stage quantum computers with limited access to quantum resources.

Use of one-dimensional CNN for input data size reduction in LSTM for improved computational efficiency and accuracy in hourly rainfall-runoff modeling

A deep learning architecture, denoted as CNNsLSTM, is proposed for hourly rainfall–runoff modeling in this study. The architecture involves a serial coupling of the one-dimensional convolutional neural network (1D-CNN) and the long short-term memory (LSTM) network. In the proposed framework, multiple layers of the CNN component process long-term hourly meteorological time series data, while the LSTM component handles short-term meteorological time series data and utilizes the extracted features from the 1D-CNN. In order to demonstrate the effectiveness of the proposed approach, it was implemented for hourly rainfall–runoff modeling in the Ishikari River watershed, Japan. A meteorological dataset, including precipitation, air temperature, evapotranspiration, longwave radiation, and shortwave radiation, was utilized as input. The results of the proposed approach (CNNsLSTM) were compared with those of previously proposed deep learning approaches used in hydrologic modeling, such as 1D-CNN, LSTM with only hourly inputs (LSTMwHour), a parallel architecture of 1D-CNN and LSTM (CNNpLSTM), and the LSTM architecture, which uses both daily and hourly input data (LSTMwDpH). Meteorological and runoff datasets were separated into training, validation, and test periods to train the deep learning model without overfitting, and evaluate the model with an independent dataset. The proposed approach clearly improved estimation accuracy compared to previously utilized deep learning approaches in rainfall = runoff modeling. In comparison with the observed flows, the median values of the Nash–Sutcliffe efficiency for the test period were 0.455–0.469 for 1D-CNN, 0.639–0.656 for CNNpLSTM, 0.745 for LSTMwHour, 0.831 for LSTMwDpH, and 0.865–0.873 for the proposed CNNsLSTM. Furthermore, the proposed CNNsLSTM reduced the median root mean square error (RMSE) of 1D-CNN by 50.2%–51.4%, CNNpLSTM by 37.4%–40.8%, LSTMwHour by 27.3%–29.5%, and LSTMwDpH by 10.6%–13.4%. Particularly, the proposed CNNsLSTM improved the estimations for high flows (≧75th percentile) and peak flows (≧95th percentile). The computational speed of LSTMwDpH is the fastest among the five architectures. Although the computation speed of CNNsLSTM is slower than LSTMwDpH's, it is still 6.9–7.9 times faster than that of LSTMwHour. Therefore, the proposed CNNsLSTM would be an effective approach for flood management and hydraulic structure design, mainly under climate change conditions that require estimating hourly river flows using meteorological datasets.

Dynamic Sensor Scheduling for Data Size Reduction in a Sensor Cloud System Based on Minimum Reconstruction Error

Dynamic Sensor Scheduling for Data Size Reduction in a Sensor Cloud System Based on Minimum Reconstruction Error

From Artificially Real to Real: Leveraging Pseudo Data from Large Language Models for Low-Resource Molecule Discovery

Molecule discovery serves as a cornerstone in numerous scientific domains, fueling the development of new materials and innovative drug designs. Recent developments of in-silico molecule discovery have highlighted the promising results of cross-modal techniques, which bridge molecular structures with their descriptive annotations. However, these cross-modal methods frequently encounter the issue of data scarcity, hampering their performance and application. In this paper, we address the low-resource challenge by utilizing artificially-real data generated by Large Language Models (LLMs). We first introduce a retrieval-based prompting strategy to construct high-quality pseudo data, then explore the optimal method to effectively leverage this pseudo data. Experiments show that using pseudo data for domain adaptation outperforms all existing methods, while also requiring a smaller model scale, reduced data size and lower training cost, highlighting its efficiency. Furthermore, our method shows a sustained improvement as the volume of pseudo data increases, revealing the great potential of pseudo data in advancing low-resource cross-modal molecule discovery.

Direct slicing of microcellular structures for digital light processing (DLP) additive manufacturing

PurposeAdditive Manufacturing (AM) conventionally necessitates an intermediary slicing procedure using the standard tessellation language (STL) data, which can be computationally burdensome, especially for intricate microcellular architectures. This study aims to propose a direct slicing method tailored for digital light processing-type AM processes for the efficient generation of slicing data for microcellular structures.Design/methodology/approachThe authors proposed a direct slicing method designed for microcellular structures, encompassing micro-lattice and triply periodic minimal surface (TPMS) structures. The sliced data of these structures were represented mathematically and then convert into 2D monochromatic images, bypassing the time-consuming slicing procedures required by 3D STL data. The efficiency of the proposed method was validated through data preparations for lattice-based nasopharyngeal swabs and TPMS-based ellipsoid components. Furthermore, its adaptability was highlighted by incorporating 2D images of additional features, eliminating the requirement for complex 3D Boolean operations.FindingsThe direct slicing method offered significant benefits upon implementation for microcellular structures. For lattice-based nasopharyngeal swabs, it reduced data size by a factor of 1/300 and data preparation time by a factor of 1/8. Similarly, for TPMS-based ellipsoid components, it reduced data size by a factor of 1/60 and preparation time by a factor of 1/16.Originality/valueThe direct slicing method allows for bypasses the computational burdens associated with traditional indirect slicing from 3D STL data, by directly translating complex cellular structures into 2D sliced images. This method not only reduces data volume and processing time significantly but also demonstrates the versatility of sliced data preparation by integrating supplementary features using 2D operations.

Energy-Efficient De-Duplication Mechanism for Healthcare Data Aggregation in IoT

The rapid development of the Internet of Things (IoT) has opened the way for transformative advances in numerous fields, including healthcare. IoT-based healthcare systems provide unprecedented opportunities to gather patients’ real-time data and make appropriate decisions at the right time. Yet, the deployed sensors generate normal readings most of the time, which are transmitted to Cluster Heads (CHs). Handling these voluminous duplicated data is quite challenging. The existing techniques have high energy consumption, storage costs, and communication costs. To overcome these problems, in this paper, an innovative Energy-Efficient Fuzzy Data Aggregation System (EE-FDAS) has been presented. In it, at the first level, it is checked that sensors either generate normal or critical readings. In the first case, readings are converted to Boolean digit 0. This reduced data size takes only 1 digit which considerably reduces energy consumption. In the second scenario, sensors generating irregular readings are transmitted in their original 16 or 32-bit form. Then, data are aggregated and transmitted to respective CHs. Afterwards, these data are further transmitted to Fog servers, from where doctors have access. Lastly, for later usage, data are stored in the cloud server. For checking the proficiency of the proposed EE-FDAS scheme, extensive simulations are performed using NS-2.35. The results showed that EE-FDAS has performed well in terms of aggregation factor, energy consumption, packet drop rate, communication, and storage cost.

Granite Exposure Mapping Through Sentinel‐2 Visible and Short Wave Infrared Bands

AbstractNonmetallic minerals like granite and limestone have calcite and biotitic ingredients as their major part which exhibit wonderful absorption features in the visible and short wave range of the electromagnetic spectrum. This research puts emphasis on delineating granite and limestone deposits of the Mardan district through the latest multispectral Landsat‐9 and Sentinel‐2 sensors of which the latter provided 94% mapping accuracy in delineating granites (biotitic bearing minerals) and limestone (calcite‐bearing minerals). The Image processing techniques of minimum noise fraction, which is double cascaded principal components analysis and pixel purity index algorithms proved helpful to bring significant improvements in classification results and in the reduction of noise and data size. The outcomes of the research study show that supervised machine learning algorithms are impactful to map such minerals provided that the data must be well organized and limited in size. The results achieved were verified through validation steps like, (a) Independent reference data of high‐resolution Google Earth maps and (b) Ground survey reports. Arc GIS 10.2 and Envi 5.3 software suite were used to create (a) ground truth points at random for accuracy assessment (b) portraying study area maps (c) Image Processing and Preprocessing tools and (d) implementation of machine learning algorithms. Access to the data and software suite is being provided for open research work.