Reduce Data Size Research Articles

High Dimensional Data Reduction in Modal Analysis with Stochastic Subspace Identification

Subspace system identification methods are widely used in output-only vibration analysis of civil structures, known as operational modal analysis. With the advent of new sensor technologies, such as video camera-based full field displacement or velocity measurements, the number of measured outputs is quickly increasing. In this paper, we propose principal component analysis-based data size reduction methods for efficient application of subspace methods, while preserving the high spatial resolution of the identified mode shapes for detailed modal analysis.

Three-dimensional reconstructing undersampled photoacoustic microscopy images using deep learning

Spatial sampling density and data size are important determinants of the imaging speed of photoacoustic microscopy (PAM). Therefore, undersampling methods that reduce the number of scanning points are typically adopted to enhance the imaging speed of PAM by increasing the scanning step size. Since undersampling methods sacrifice spatial sampling density, by considering the number of data points, data size, and the characteristics of PAM that provides three-dimensional (3D) volume data, in this study, we newly reported deep learning-based fully reconstructing the undersampled 3D PAM data. The results of quantitative analyses demonstrate that the proposed method exhibits robustness and outperforms interpolation-based reconstruction methods at various undersampling ratios, enhancing the PAM system performance with 80-times faster-imaging speed and 800-times lower data size. The proposed method is demonstrated to be the closest model that can be used under experimental conditions, effectively shortening the imaging time with significantly reduced data size for processing.

Optimizing Error-Bounded Lossy Compression for Scientific Data With Diverse Constraints

Vast volumes of data are produced by today's scientific simulations and advanced instruments. These data cannot be stored and transferred efficiently because of limited I/O bandwidth, network speed, and storage capacity. Error-bounded lossy compression can be an effective method for addressing these issues: not only can it significantly reduce data size, but it can also control the data distortion based on user-defined error bounds. In practice, many scientific applications have specific requirements or constraints for lossy compression, in order to guarantee that the reconstructed data are valid for post hoc analysis. For example, some datasets contain irrelevant data that should be isolated in particular and users often have intuition regarding value ranges, geospatial regions, and other data subsets that are crucial for subsequent analysis. Existing state-of-the-art error-bounded lossy compressors, however, do not consider these constraints during compression, resulting in inferior compression ratios with respect to user's post hoc analysis, due to the fact that the data itself provides little or no value for post hoc analysis. In this work we address this issue by proposing an optimized framework that can preserve diverse constraints during the error-bounded lossy compression, e.g., cleaning the irrelevant data, efficiently preserving different precision for multiple value intervals, and allowing users to set diverse precision over both regular and irregular regions. We perform our evaluation on a supercomputer with up to 2,100 cores. Experiments with six real-world applications show that our proposed diverse constraints based error-bounded lossy compressor can obtain a higher visual quality or data fidelity on reconstructed data with the same or even higher compression ratios compared with the traditional state-of-the-art compressor SZ. Our experiments also demonstrate very good scalability in compression performance compared with the I/O throughput of the parallel file system.

Data reduction in fog computing and internet of things: A systematic literature survey

Cloud computing, the most crucial computing and storage tool in IoT (Internet of Things), still meets various challenges. The remoteness of IoT end devices from cloud platforms may lead to significant issues for real-time applications such as disaster handling, healthcare application, etc. In order to address these issues, fog computing is a new platform with specialized features that perform essential IoT data management and real-time application management tasks. Data management in IoT via fog computing is critical for decreasing latency in real-time IoT applications and is needed to produce more professional knowledge and smart decisions. Reduction in the size of data sent to the cloud layer is a fundamental topic in data management on the fog computing platform. In this paper, to select and survey studies about data size reduction in fog computing, we applied the Systematic Literature Review (SLR) process to comprehend and classify the different topics and related approaches in this field. In addition, the studies presented in the edge computing field, which were close to our goal, were also investigated. The primary purpose of this study is to classify and analyze the fog data reduction (FDR) studies published between 2016 and 2022. The topics and related approaches of selected papers are presented in an approach-based taxonomy. The topics of provided taxonomy include data filtering, data compression, data aggregation, data prediction, data pattern recognition, and general facets and related approaches. Finally, open issues for FDR and relevant main challenges are presented in upcoming research.

Machine Learning As Seed Image Identification Using Principal Component Analysis (PCA)

Plants can be identified using several variables, such as seeds’ shapes, colors, and sizes. However, several types of plants have close similarities to seed shapes. Therefore, additional characteristics are necessary to support the identification process. This study applied machine learning with the PCA method to identify plant species from seed shapes. The PCA simplifies the observed variables by reducing data dimensions and storing 75% of the information. The procedure did not eliminate too much important information while reducing data size and processing time. We collected 100 images of plant seeds similar to one another, such as sapodilla seeds, soursop, cucumber, star fruit, grape, melon, apple, lime, watermelon, and chili. A measurement system was designed using the K-Fold Cross Validation, and 10 tables of experimental results discovered a good level of accuracy of 83%. The Omission error occurred in the seeds of soursop, starfruit, grape, apple, lime, and watermelon while the most commission errors occurred in apple seeds (8 times).

Physics-Enhanced PCA for Data Compression in Edge Devices

In smart cities, tremendous data are generated with edge devices continuously for scientific applications, such as structural health monitoring (SHM), leading to a high bandwidth burden to edge devices. Data compression is a typical technique for reducing data size and improving transmission efficiency. However, their operations for distinguishing redundant data might be unapplicable for a domain-specific scientific analysis and distort physical quantities of raw data (i.e., mode shapes of vibration data). Besides, they are too compute-intensive to be executed in resource-constraint edge devices. In this paper, we leverage physical knowledge to enhance a lightweight data compression method for edge devices in maintaining physical quantities. In particular, we propose physics-enhanced PCA for compressing data in a dynamic system — compressing the vibration data of a structure in the context of SHM. Physical knowledge is identified from a structure and guides the compression process to preserve the mode shape, an significant physical quantity for structures. We have formally analyzed the effectiveness of our approach and performed experiments to show that physical knowledge is essential for preserving mode shapes. Concretely, experiments in numerical and real-world structures show that physics-enhanced PCA can improve the accuracy by up to 56% compared with alternative baseline.

A secure IoT based healthcare framework using modified RSA algorithm using an artificial hummingbird based CNN

AbstractRecently, a wide variety of real‐world problems are solved via cloud computing, deep learning, machine learning, artificial intelligence, and the Internet of Things (IoT). These methodologies are concerned with different areas like smart cities, agriculture, transportation systems, and healthcare systems. The existing researchers focused on the health care monitoring application along with IoT and cloud computing. It met several shortcomings in case of computational complexity, cost, time, and improper health care data storage and so forth. To overcome these challenges, the novel IoT‐enabled secure healthcare monitoring model is proposed in this study. At first, several sensors are deployed in the human body thereby collecting patients' data with respect to vital parameters like body temperature deviation. The patient's health record dataset consists of 10 attributes namely phone numbers, marital status, address, age, name, heartbeat rate, oxygen level, smoking, temperature, and blood pressure. The data size reduction and normalization are performed in the IoT medical sensor dataset which contains the redundant or irrelevant attributes that are eliminated during pre‐processing. The artificial hummingbird (AHB) algorithm‐based convolutional neural network (AHB‐CNN) model performs both feature extraction and classification of cancer disease. The AHB‐CNN model classifies whether the patient is prone to cancer or not based on the sensor input collected. The received results are then sent to the hospital management for analysis. The Rivest‐Shamir‐Adleman (RSA) encryption method is mostly used in this study due to the major benefits it gives in terms of asymmetric encryption, ease of use, simpler deployment, and high security associated with factoring large prime numbers. A modified RSA algorithm is used in this article which uses the double encryption‐decryption process and “n” prime numbers to enhance the security of the conventional RSA algorithm. The data is always encrypted during transit before being stored in the cloud or for further processing. Depending upon the experimental consequences, the proposed method established superior performances compared to other state‐of‐art techniques.

A reinforced active learning approach for optimal sampling in aspect term extraction for sentiment analysis

Aspect level sentiment analysis is a fine grained task in sentiment analysis which identifies the product features from an opinionated piece of text and maps the sentiment towards each of them. Supervised ML algorithms have reported comparatively higher performance on aspect level sentiment analysis but at the cost of substantial qualitative labelled data. Data labelling for such fine grained tasks also demand domain knowledge and expertise. Hence a mechanism to extract a minimal informative subset which is almost representative of the entire data would be a breakthrough in bringing down the annotation costs to a large extent. The proposed methodology puts forward an active learning based sampling strategy for aspect term extraction, a subtask in aspect level sentiment analysis which identifies the product features. The sampling strategy is automated by reinforcement learning which extracts an optimal sample from the entire unlabelled training data and hence optimizes data annotation by reducing the time and effort linked to the labelling process. This work is of high importance in a data driven era where companies invest a lot in collecting and annotating huge volumes of data. The model has been experimented across the laptop and restaurant domains of SemEval (2014–2016) datasets. The experiments proved that a considerable reduction of the training data size is achieved across different datasets. The model trained on the data extracted by the proposed reinforced active learning model beats random sampling by 9 to 17 points when evaluated on the F-measure of the extracted aspect terms and is almost on par with the model trained on the entire training data by utilising hardly 9 to 13% of the entire training data across the datasets experimented.

Performance Analysis of Hybrid Cryptography System for High Security and Cloud-Based Storage

Now a day, the security of the data is playing a major part in communication systems due to further bushwhackers between channels media. The security position depends on cache crucial and as per literature, advanced the bit size of the key, advanced the security and also larger data size comes major challenge task for the further process. Thus, the generation of crucial with the further size is a major grueling task and at present, the Advanced Encryption Standard (AES) is a better cryptography system where the encryption and decryption can perform with a fixed key. The literature says the holomorphic function is well-suitable for data size reduction. To address this issue, new Holomorphic grounded encryption and decryption and AES are combined to increase the security position. The alternate novelty is that variable crucial generation using Elliptic Wind Cryptography (ECC) due to its enlarged proportion of consideration in assiduity and experimenters. The ECC uses point addition and point doubling to induce 256 values and addition operations can be avoided. After the generation of the matrix, each matrix value is translated and decrypted using a Holomorphic algorithm. The proposed work has been designed using MATLAB 2017a, dissembled, and validated with different datasets in real decors. Cloud computing is expected to be considered one of the primary computing platforms in the field of storage and security as it possesses many advantages such as profitability, efficiency as well as lower implementation overheads. Contemporary cloud computing security algorithms are enhanced extensions of cryptography. Data privacy, as well as data protection, are the major areas of concern in Cloud computing. The cryptographic with holomorphic based data encryption and interchange of information is exchanged and then accumulated in the cloud through holomorphic encryption which uses point addition and doubling operation to ensure data confidentiality of owners as well as users. Proposed work novel hybrid algorithm based on the context of encryption and decryption and thus integrates cryptography hybrid techniques include modified 126-bit AES and ElGamal based ECC through splitting algorithm. The advantage of splitting the larger data in size into binary form and then processing for encryption and decryption leads to optimization of latency, increase throughput, and security. The proposed hybrid approach has better security towards information sharing as well as cloud storage intrusions. Based on obtained results in MATLAB 2017a software tool, the obtained results show that 43% improvement in throughput and 12% reduction in latency, and a 21% improvement in security level.

A structural monitoring data processing model based on signal musicalization

Signal features extraction is a significant issue during the damage assessment in structural health monitoring (SHM). Multiple sensor arrays would generate data redundancy in long-term monitoring. It is a challenge to extract signal features and reduce data size. For extracting signal features with low storage consumption, this research proposed a monitoring data processing method based on signal musicalization. First, the time domain signal is sampled by music spectrum to complete the mapping from imperceptible signal to audible audio. The damage signal feature is encoded into musical indexes suitable for serialized storage. Then, the damage modal identification based on musicalized indexes is completed using Long Short-Term Memory (LSTM) network. Further, a data recovery mechanism for failure sensor based on Kalman algorithm is presented. Experiments show that multiple sensor signal features could be extracted by this method. Moreover, it has advantages in damage identification and data compression.

Fault Classification in a DG Connected Power System using Artificial Neural Network

Distributed generation is playing an important role in power system to meet the increased load demand. Integration of Distributed Generator (DG) to grid leads to various issues of protection and control of power system structure. The effect of the distributed generators to the grid is changes the fault current level, which makes the fault analysis more complex. From the different fault issues occurs in a distributed generator integrated power system, classification of fault remains as one of the most vital issue even after years of in-depth research. This paper emphasis on the classification of faults in DG penetrated power system using Artificial Neural Network (ANN). Because researchers are attempting to detect and diagnose these faults as soon as possible in order to avoid financial losses, this work aims to investigate the sort of fault that happened in the hybrid system. This paper proposed artificial neural network based approaches for fault disturbances in a microgrid made up of wind turbine generators, fuel cells, and diesel generator. The voltage signal is retrieved at the point of common coupling (PCC). The extracted data are used for training and testing purpose. Artificial neural network technique is utilized for the classification of fault in the simulated model. Furthermore, performance indices (PIs) such as standard deviation and skewness are calculated for reduction of data size and better accuracy. Both the fault and parameters are varied to check the usefulness of the proposed method. Finally, the results are discussed and compared with different DG penetration.

The minimizer Jaccard estimator is biased and inconsistent.

MotivationSketching is now widely used in bioinformatics to reduce data size and increase data processing speed. Sketching approaches entice with improved scalability but also carry the danger of decreased accuracy and added bias. In this article, we investigate the minimizer sketch and its use to estimate the Jaccard similarity between two sequences.ResultsWe show that the minimizer Jaccard estimator is biased and inconsistent, which means that the expected difference (i.e. the bias) between the estimator and the true value is not zero, even in the limit as the lengths of the sequences grow. We derive an analytical formula for the bias as a function of how the shared k-mers are laid out along the sequences. We show both theoretically and empirically that there are families of sequences where the bias can be substantial (e.g. the true Jaccard can be more than double the estimate). Finally, we demonstrate that this bias affects the accuracy of the widely used mashmap read mapping tool.Availability and implementationScripts to reproduce our experiments are available at https://github.com/medvedevgroup/minimizer-jaccard-estimator/tree/main/reproduce.Supplementary information Supplementary data are available at Bioinformatics online.

De-aliased depth-range-extended optical coherence tomography based on dual under-sampling

We demonstrate a dual under-sampling (DUS) method to achieve de-aliased and depth-range-extended optical coherence tomography (OCT) imaging. The spectral under-sampling can significantly reduce the data size but causes well-known aliasing artifacts. A change in the sampling frequency used to acquire the interference spectrum alters the aliasing period within the output window except for the true image; this feature is utilized to distinguish the true image from the aliasing artifacts. We demonstrate that with DUS, the data size is reduced to 37% at an extended depth range of 24 mm, over which the true depth can be precisely measured without ambiguity. This reduction in data size and precise measuring capability would be beneficial for reducing the acquisition time for OCT imaging in various biomedical and industrial applications.

Identifying Potentially Risky Intersections for Heavy-Duty Truck Drivers Based on Individual Driving Styles

In developing countries, heavy-duty trucks play an important role in transportation for infrastructure construction. However, frequent truck accidents cause great losses. Previous studies have mainly focused on passenger drivers; to date, little has been done to assess the driving behavior of heavy truck drivers. The overall objective of this study is to classify driving styles at intersections, analyze the impacts of differing types of traffic control at intersections on driving styles, and identify potentially risky intersections. We selected 11 heavy-duty truck drivers and collected kinematic driving parameters (including driving speed and both lateral and longitudinal acceleration) from field experiments in Nanjing for our study. Our study on driving styles followed the following steps. First, we reduced data size and extracted data features on the basis of time windows in Python. Second, driving styles were classified into three driving styles: cautious, normal, and aggressive, based on the K-means clustering method, and the corresponding thresholds for each category were obtained. Kinematic driving parameters were used as driving style measurements. Third, according to classifications of driving style, the impacts of four different intersection traffic control types: two-phase signalized, multiphase signalized, stop, and yield intersections, on driving styles have been analyzed using the multinomial logit model. Moreover, based on the above analysis, potentially risky intersections were identified. The results suggest that different types of traffic control at intersections lead to variations in driving styles and have different influences on driving styles. In terms of accuracy, our method, which uses driving speed, both lateral and longitudinal acceleration, and jerk as features, performs better than traditional methods which only use speed and acceleration. The results of the study allow us to analyze the driving data of heavy-duty trucks and identify drivers who drive more aggressively during a trip. In addition, the results show that aggressive driving styles mostly occur at stop intersections and in the dilemma zones of signalized intersections. Therefore, early-warning interventions can be provided during a driver’s trip by analyzing the different types of traffic control at intersections on the route in advance. Finally, the cumulative analysis of driving styles at intersections over multiple trips can be used to identify potentially high-risk intersections. It is possible to eliminate potential risks in these areas through measures such as early warnings and by improving traffic management control methods.

An intuitionistic fuzzy bireduct model and its application to cancer treatment

Due to technological advancement, data size has seen a significant increase both in terms of features and instances. An efficient way to handle large sized datasets is to apply data reduction technique to ease the computational intractability of learning algorithms. Intuitionistic fuzzy rough and fuzzy rough sets based approaches provide a handful of solution, but the applicability is mostly restricted to reduction in terms of feature size. However, reduction in terms of instance size has not been handled by any of the existing intuionistic fuzzy set approaches. The present paper introduces the notion of bireducts in intuitionistic fuzzy framework that can be used for simultaneous reduction of instances and features. A robust lower approximation formulation is employed along with laying the foundation for the variants of instance selection technique. The proposed model is therefore robust to noise and can very effectively handle uncertainities due to judgement as well as identification. Further, an efficient instance selection technique in bireduct formulation enhances the performance. The experimental evaluation on benchmark datasets demonstrates the applicability and robustness of the proposed bireducts. It significantly reduces data size both in terms of instances and features whilst maintaining high performances. Further, the model is applied in the challenging domain of cancer treatment by enhancing the prediction performance of anti-angiogenic peptides. A comparative analysis demonstrates the superiority of the proposed methodology.

Reconstructing Dynamic 3D Models with Small Data by Integrating Position-Based Dynamics and PDE-Based Modelling

Simulation with position-based dynamics is very popular due to its high efficiency. However, it has the weaknesses of lacking details when too few vertices are involved in simulation and inefficiency when too many vertices are used for simulation. To tackle this problem, in this paper, we propose a new method of reconstructing dynamic 3D models with small data. The core elements of the proposed approach are a curve-represented geometric model and a physics-based mathematical model defined by dynamic partial differential equations. We first use the simulation method of position-based dynamics to generate a group of keyframe poses, which are used to create the deformation animation of a 3D model. Then, wireframe curves are extracted from skin deformation shapes of the 3D model at different keyframe poses. A physics-based mathematical model defined by dynamic partial differential equations is proposed. Its closed-form solution is obtained to represent the extracted curves, which are used to reconstruct the deformation models at different keyframe poses. Experimental examples and comparisons made in this paper indicate that the proposed method of reconstructing dynamic 3D models can greatly reduce data size while keeping good details.

Level-Crossing Sampling with Multiple Temporal Resolutions for Speech Signals

Level-crossing sampling (LCS) is an alternative to the uniform sampling method and is suitable for sparse and bursty signals. LCS, which is adaptive to amplitude variations of the signals, mainly focuses on reducing the total number of samples. Presently, speech signals are sampled with LCS methods that have a constant temporal resolution, and they require time-to-digital converters (TDC) with a wide dynamic range. A long sequence of overflow counts of the TDC counter may be generated during silence regions if the dynamic range is reduced. This paper proposes level-crossing samplers with multiple temporal resolutions for speech signals. Frequency scaling is applied to standard LCS, adaptive LCS and peak sampling. During the silence region, the frequency of the TDC clock is scaled down, which enables the representation of longer intervals with smaller words. The number of bits per sample and hence the total number of samples generated are decreased due to the reduction in the dynamic range of the TDC. Simulation results indicate that the proposed method outperforms the conventional level-crossing samplers, which do not employ frequency scaling. In addition to a significant reduction in data size, this method can also be used for real-time automatic detection of silence regions in a speech signal.

A K-Means clustering and SVM based hybrid concept drift detection technique for network anomaly detection

Today’s internet data primarily consists of streamed data from various applications like sensor networks, banking data and telecommunication data networks. A new field of study, data stream mining has been gaining popularity to study streamed data behavior. Detection of anomalies in the network traffic also finds its applicability in this context. However traditional machine learning algorithms suffer in providing consistent high accuracy values and give high false alarms. This is due to the presence of concept drift in the captured data streams. Concept drift describes unknown changes in the characteristics of network data over time. Therefore, to handle presence concept drift new methodologies and techniques for drift detection, understanding and adaptation are required. In this paper, we have proposed two techniques, an Error Rate Based Concept Drift Detection and Data Distribution Based Concept Drift Detection and studied their impact. Furthermore, sliding window based data capturing and drift analyzing combined with K-Means Clustering has been used for reducing data size and upgrading training dataset. We have used the Support Vector Machine (SVM) classifier for anomaly detection and retraining of the model has been initiated based on statistical tests. The experiments have been performed on three datasets, namely, generated Testbed Dataset, NSL-KDD and CIDDS-2017. Detection accuracy, KL-Divergence and Kappa Statistics have been used to study the severity of the concept drift in the datasets. After applying the proposed approach, the SVM has been shown to have a better classification accuracy of 93.52%, 99.80% and 91.33% respectively. We achieved a precision rate of 91.84%, 99.1% and 88.3%, a recall rate of 94.30%, 99.2% and 91.7% with an F1 score of 92.9%, 99.15% and 89.6% respectively.

An Effective Foveated 360° Image Assessment Based on Graph Convolution Network

Virtual reality (VR) has been adopted in various fields such as entertainment, education, healthcare, and the military, due to its ability to provide an immersive experience to users. However, 360° images, one of the main components in VR systems, have bulky sizes and thus require effective transmitting and rendering solutions. One of the potential solutions is to use foveated technologies, that take advantage of the foveation feature of the human eyes. Foveated technologies can significantly reduce the data required for transmission and computation complexity in rendering. However, understanding the impact of foveated 360° images on human quality perception is still limited. This paper addresses the above problems by proposing an accurate machine-learning-based quality assessment model for foveated 360° images. The proposed model is proven to outperform the three cutting-edge machine-learning-based models, which apply deep learning techniques and 25 traditional-metric-based models (or analytical-function-based-models), which utilize analytical functions. It is also expected that our model helps to evaluate and improve 360° content streaming and rendering solutions to further reduce data sizes while ensuring user experience. Also, this model could be used as a building block to construct quality assessment methods for 360° videos, that are reserved for our future work. The source code is available at https://github.com/telagment/FoVGCN.

ANALYSIS OF IN-PLACE MATRIX ROTATION OF SQUARE MATRIX FOR INFORMATION SECURITY APPLICATIONS

The concept of matrices was developed to represent complex data into understandable and formattable data. There is a huge number of applications in which matrices are used like classification and data size reduction used to store it. Matrices are widely used in most cryptographic algorithms. Every image can be represented as a matrix where each pixel value is stored as an element of a matrix. The rotation of the matrix defines that a matrix is rotated in the XY-plane to a certain angle concerning the x-axis about the origin. A rotation matrix is a transformation matrix in linear algebra that is used to make a rotation in Euclidean space. The rotation of the matrix includes transformation such that it requires additional memory for a new transformed value. This paper describes the in-place matrix rotation of square matrix using 45° rotation of information. In-place algorithm alters the input without requiring additional memory. A little amount of additional memory could be needed for the implementation of an in-place algorithm. However, the amount of memory required should be consistent and not dependent on the size of the input. With the help of In-place Matrix Rotation (IMR) of the complementary angle, the IMR of 45° is proved. In IMR, the elements change their positions, but the elements remain the same in the matrix. Using IMR is the matrix dimensions are changed with the center fixed to a particular position just by rotating the matrix not transforming the position.