Amount Of Data Storage Research Articles

Recursive Delta-Operator-Based Subspace Identification with Fixed Data Size

This paper proposes a recursive delta-operator-based subspace identification method with fixed data size. The majority of existing subspace identification methods are constrained to discrete-time systems because of the disparity in Hankel matrices. Additionally, due to the storage cost, LQ-factorization and singular value decomposition in identification methods are best suited for batch processing rather than online identification. The continuous-time systems are transformed into state space models based on the delta-operator to address these issues. These models approach the original systems when the sampling interval approaches zero. The size of the data matrices is fixed to reduce the computing load. By fading the impact of past data on future data, the amount of data storage can be decreased. The effectiveness of the proposed method is illustrated by the continuous stirred tank reactor system.

TCMQMS System: A digital twin and blockchain-based platform for tracing the whole process of Chinese medicine quality information

Traditional Chinese medicine is an important medicine and health food in China. In the process of making Chinese medicines, the main active ingredients of the medicines are highly dependent on the production environment due to the special physical characteristics of the medicines themselves. Any slight change in the environment can lead to changes in the active ingredients as well as the final efficacy, even affecting the patient treatment cycle, and this characteristic is not conducive to quality control by national drug quality regulators. In order to enhance the quality management of TCM, a blockchain and digital twin based whole process monitoring system for TCM quality is proposed: Traditional Chinese Medicine Quality Monitoring System (TCMQMS), in order to achieve a whole process, full scope and highly transparent TCM quality monitoring model for TCM production from the source of cultivation to the final patient. And. A blockchain platform based on Fabric blockchain data development platform as well as Sia distributed data storage was designed.
 With the help of Sia distributed storage technology, the amount of data storage is significantly compressed, which achieves the purpose of unifying the huge information data flow into the TCMQMS system and quickly building the system platform using the Fabric platform. In addition, in order to ensure that data is read and written in real time during the manufacturing and transportation of Chinese medicine, environmental sensors, such as temperature and humidity sensors, are placed in the Chinese medicine processing plant and in the transport sector. The environmental sensors, such as temperature and humidity sensors, are placed inside the Chinese medicine processing plants and transport vehicles. The environmental data is combined with the production data recorded by the TCMQMS system and sent to the digital twin TCM manufacturing simulation environment in real time, and through the simulation of the TCM manufacturing environment, the TCM quality environment parameters are continuously iteratively updated to be more suitable for production. This is coupled with real-time linkage with environmental sensors to achieve real-time adjustment of environmental parameters. In addition, the TCMQMS system records the entire process data, giving each data block a unique and tamper-proof hash value and CA certificate, so that when there is a possible quality problem, the hash value and CA certificate will be used to quickly trace the person responsible and the details of the environment in which the TCM was produced in real time. Finally, we present a real-life example of the use of the TCMQMS system to validate our proposed approach.

Distributed Data Integrity Verification Scheme in Multi-Cloud Environment.

Most existing data integrity auditing protocols in cloud storage rely on proof of probabilistic data possession. Consequently, the sampling rate of data integrity verification is low to prevent expensive costs to the auditor. However, in the case of a multi-cloud environment, the amount of stored data will be huge. As a result, a higher sampling rate is needed. It will also have an increased cost for the auditor as a consequence. Therefore, this paper proposes a blockchain-based distributed data integrity verification protocol in multi-cloud environments that enables data verification using multi-verifiers. The proposed scheme aims to increase the sampling rate of data verification without increasing the costs significantly. The performance analysis shows that this protocol achieved a lower time consumption required for verification tasks using multi-verifiers than a single verifier. Furthermore, utilizing multi-verifiers also decreases each verifier's computation and communication costs.

Automated multimodal fluorescence microscopy for hyperplex spatial-proteomics: Coupling microfluidic-based immunofluorescence to high resolution, high sensitivity, three-dimensional analysis of histological slides.

In situ multiplexing analysis and in situ transcriptomics are now providing revolutionary tools to achieve the comprehension of the molecular basis of cancer and to progress towards personalized medicine to fight the disease. The complexity of these tasks requires a continuous interplay among different technologies during all the phases of the experimental procedures. New tools are thus needed and their characterization in terms of performances and limits is mandatory to reach the best resolution and sensitivity. We propose here a new experimental pipeline to obtain an optimized costs-to-benefits ratio thanks to the alternate employment of automated and manual procedures during all the phases of a multiplexing experiment from sample preparation to image collection and analysis. A comparison between ultra-fast and automated immunofluorescence staining and standard staining protocols has been carried out to compare the performances in terms of antigen saturation, background, signal-to-noise ratio and total duration. We then developed specific computational tools to collect data by automated analysis-driven fluorescence microscopy. Computer assisted selection of targeted areas with variable magnification and resolution allows employing confocal microscopy for a 3D high resolution analysis. Spatial resolution and sensitivity were thus maximized in a framework where the amount of stored data and the total requested time for the procedure were optimized and reduced with respect to a standard experimental approach.

Overlapping Reconstructions in Thin-section Computed Tomography: Benefits for Lung Nodule Volume Measurements.

The aim of our study was to assess the influence of overlapping image reconstruction on thin-section chest computed tomography (CT) in patients with small lung nodules. In all, 40 patients with 128 pulmonary nodules underwent chest CT on a third-generation dual-source CT. All images were reconstructed with a section thickness of 1 mm and an increment of 0.5 to 1 mm. Two readers performed volume measurements with a commercially available software package. Our data indicate no relevant differences (P=0.44) for volume measurements of solid lung nodules comparing overlapping (50% overlap) and nonoverlapping image reconstructions with a mean difference of 0.5±2.3 mm3 and a mean absolute difference of nodule volume of 2±2% (range, 0% to 11%). Interobserver differences were 1.0±3.7 mm3 (reconstruction with 50 overlap) and 1.1±4.1 mm3 (nonoverlapping reconstructions). Differences in nodule volume measurements between overlapping and nonoverlapping image reconstructions were similar compared with interobserver differences. Obviating the reconstruction of overlapping sections for lung nodule volume measurements would reduce the duration of image reconstruction and amount of data storage.

A low-altitude public air route network for UAV management constructed by global subdivision grids.

With an increasing number of unmanned aerial vehicles (UAVs), the difficulty of UAV management becomes more challenging, especially for low-altitude airspace due to complicated issues of security, privacy and flexibility. Existing management approaches to UAV flights include implementing registration of flight activity for supervision purposes, limiting the maximum flight height, setting different zones for different flight activities and prohibiting flights. In this research, we proposed a new air traffic management method for UAVs based on global subdivision theory. We designed four types of low-altitude air routes from grids, which correspond to grid sizes of 1.85 km, 128 m, 64 m and 32 m. Utilization of the subdivision grids transforms the complex spatial computation problem into a query process in the spatial database, which provides a new approach to UAV management in the fifth-generation (5G) era. We compared the number and data size of stored track records using longitude and latitude and different grid levels, computed time consumption for air route trafficability and simulated UAV flight to verify the feasibility of constructing this type of air traffic highway system. The amount of data storage and time consumption for air route trafficability can be substantially reduced by subdivision. For example, the data size using traditional expressions of latitude and longitude is approximately 1.5 times that of using a 21-level grid, and the time consumption by coordinates is approximately 1.5 times that of subdivision grids at level 21. The results of the simulated experiments indicate that in the 5G environment, gridded airspace can effectively improve the efficiency of UAV trajectory planning and reduce the size of information storage in the airspace environment. Therefore, given the increasing number of UAVs in the future, gridded highways have the potential to provide a foundation for various UAV applications.

Research on the Attribute Extraction Algorithm of People and Vehicles Based on Video Structure

With the development of sensor technology and computer vision, the speed of video generation is increasing exponentially. The workload of video browsing and the amount of stored data are increasing. In order to achieve fast browsing and efficient storage, video structure technology is increasingly important. This paper focuses on the video structure technology in the construction of smart cities, outlines the system process of video structure, and studies the extraction algorithm of human and vehicle target attributes based on deep learning. And combined with the actual video monitoring scene, the experiment is carried out. Compared with traditional image processing methods, the video structure target attribute extraction model based on deep learning has high detection accuracy and strong algorithm generalization ability. The intelligent analysis and processing of data by video structure technology is conducive to quickly increasing the speed of target search, reducing storage capacity, solving the problem of long-term storage, and realizing full automation of surveillance video.

An improved material point method for coining simulation

Extremely small time step caused by large deformation in coining process significantly increases computational time in dynamic explicit finite element method (FEM). Furthermore, negative volumes of distortion elements in the rim of coin lead to abnormal termination of simulation. In order to overcome these problems, dynamic explicit material point method (MPM) is employed for coining process. However, most of the point-to-point contact algorithms in MPM cause fake contacting detection. In addition, abundant tiny patterns on die surfaces put forward high requirement for the contact algorithm to calculate the normal vectors. A point-to-segment based contact algorithm is proposed to obtain accurate normal vectors at real physical contact point and embedded into the framework of modified update stress last (MUSL) for simulating coining. Amount of data storage and computations are reduced since the background domain only contains the deformable workpiece and one velocity field is required. The improved MPM is applied to predict insufficient filling during minting, and its performance is validated by experimental results.

A Framework for Executing Complex Querying for Relational and NoSQL Databases (CQNS)

The increasing of data on the web poses major confrontations. The amount of stored data and query data sources have become needful features for huge data systems. There are a large number of platforms used to handle the NoSQL database model such as: Spark, H2O and Hadoop HDFS / MapReduce, which are suitable for controlling and managing the amount of big data. Developers of different applications impose data stores on difficult tasks by interacting with mixed data models through different APIs and queries. In this paper, a complex SQL Query and NoSQL (CQNS) framework that acts as an interpreter sends complex queries received from any data store to its corresponding executable engine called CQNS. The proposed framework supports application queries and database transformation at the same time, which in turn speeds up the process. Moreover, CQNS handles many NoSQL databases like MongoDB and Cassandra. This paper provides a spark framework that can handle SQL and NoSQL databases. This work also examines the importance of MongoDB block sharding and composition. Cassandra database deals with two types of sections vertex and edge Portioning. The four scenarios criteria datasets are used to evaluate the proposed CQNS to query the various NOSQL databases in terms of optimization performance and timing of query execution. The results show that among the comparative system, CQNS achieves optimum latency and productivity in less time.

Contributions to the adaptive Monte Carlo method

The GUM Supplement 1 presented the adaptive Monte Carlo (AMC) method. A basic implementation of an AMC procedure involves carrying out an increasing number of Monte Carlo trials until four parameters have stabilized in a statistical sense. Although the AMC method has been successfully used for uncertainty evaluations, the amount of stored data was seen as significant, and even after achieving stability, it can be lost with the increase in the number of trials. To overcome these problems, two modifications to the AMC method were proposed, implemented and validated. The first is related to data storage, while the second consists of applying an alternative criterion to assess convergence. After modifications, the AMC was named the modified adaptive Monte Carlo (MAMC) method and was applied when estimating the uncertainty of measurements carried out with a micrometer. The MAMC effectiveness was validated through the comparison of the uncertainty values and those from the application of the GUM and AMC methods. Under the evaluated experimental conditions, the MAMC showed greater repeatability when compared to AMC, regarding the number of trials to be carried out. This factor contributes toward the higher reliability of this method. The amount of data to be stored and manipulated through the application of the MAMC method was decreased significantly. This fact may increase the adoption of the AMC method.

Feedback-Dubins-RRT Recovery Path Planning of UUV in an Underwater Obstacle Environment

In this paper, a UUV (Unmanned Underwater Vehicle) recovery path planning method of a known starting vector and end vector is studied. The local structure diagram is designed depending on the distance and orientation information about the obstacles. According to the local structure diagram, a Rapidly exploring Random Tree (RRT) method with feedback is used to generate a 3D Dubins path that approaches the target area gradually, and the environmental characteristic of UUV reaching a specific target area is discussed. The simulation results demonstrate that this method can effectively reduce the calculation time and the amount of data storage required for planning. Meanwhile, the smooth spatial path generated can be used further to improve the feasibility of the practical application of UUV.

Code health in EOS: Improving test infrastructure and overall service quality

During the last few years, the EOS distributed storage system at CERN has seen a steady increase in use, both in terms of traffic volume as well as sheer amount of stored data. This has brought the unwelcome side effect of stretching the EOS software stack to its design constraints, resulting in frequent user-facing issues and occasional downtime of critical services. In this paper, we discuss the challenges of adapting the software to meet the increasing demands, while at the same time preserving functionality without breaking existing features or introducing new bugs. We document our efforts in modernizing and stabilizing the codebase, through the refactoring of legacy code, introduction of widespread unit testing, as well as leveraging Kubernetes to build a comprehensive test orchestration framework capable of stressing every aspect of an EOS installation, with the goal of discovering bottlenecks and instabilities before they reach production.

An Efficient Blockchain-Based Bidirectional Friends Matching Scheme in Social Networks

In social networks, the personal attributes or hobbies of the users are exposed to the server to establish the relationships. Service providers may store these information for commercial purpose or statistical analysis. Furthermore, the server may expose to external attacks, which may disclose users' privacy information. In this paper, we present a hierarchical blockchain-based attribute matching scheme, which realizes privacy-preserving attribute matching under multiple semi-trusted servers. The scheme employs CP-ABE and bloom filter to satisfy the requirements of the users to make friend discovery, and reduces the computation cost of users by outsourcing decryption of CP-ABE. Besides, the hierarchical blockchain only implements the consensus and storage of matching results on the blockchain, while the complex calculations and a large amount of data storage are off-chain, which reduces the consumption of the blockchain and improves the operation efficiency. Finally, we prove the scheme can resist single point failure, collusion attack, internal attack and external attack, the experimental results demonstrate the proposed scheme is feasibility and efficiency.

Research of NOSQL and SQL and Designing a Better Database Schema for Databases

The amount of data storage and querying is increasing day by day. The requirement of data-storage, security, scalability and management is an alarming issue. Structured Query Language (SQL) databases was designed to overcome most of the trouble faced over tradition file storage system. But the alarming rate of data storage leads to Not only Structured Query Language (NoSQL) databases. in this paper, various aspects of SQL and NoSQL are compared with respect to their data management and to develop a better schema for data management is attempted.

Gridless Parameter Estimation for One-Bit MIMO Radar With Time-Varying Thresholds

We investigate the one-bit MIMO (1b-MIMO) radar that performs one-bit sampling with a time-varying threshold in the temporal domain and employs compressive sensing in the spatial and Doppler domains. The goals are to significantly reduce the hardware cost, energy consumption, and amount of stored data. The joint angle and Doppler frequency estimations from noisy one-bit data are studied. By showing that the effect of noise on one-bit sampling is equivalent to that of sparse impulsive perturbations, we formulate the one-bit $\ell_1$-regularized atomic-norm minimization (1b-ANM-L1) problem to achieve gridless parameter estimation with high accuracy. We also develop an iterative method for solving the 1b-ANM-L1 problem via the alternating direction method of multipliers. The Cram$\acute{\text{e}}$r-Rao bound (CRB) of the 1b-MIMO radar is analyzed, and the analytical performance of one-bit sampling with two different threshold strategies is discussed. Numerical experiments are presented to show that the 1b-MIMO radar can achieve high-resolution parameter estimation with a largely reduced amount of data.

A self-adaptive method for the determination of solar bursts for high-resolution solar radio spectrometer

The solar radio spectrum analyzer is the main means of observing solar radio bursts. Although increasing the temporal resolution allows the fine structure of the solar radio spectrum to be observed in more detail, it also increases exponentially the amount of data received, thereby creating difficulties for the receiving system. To reduce the amount of data handled by the high-time-resolved spectrometer when the Sun is quiet, this paper proposes a self-adaptive method for determining solar bursts by selecting 20 frequencies for radio flux profiles. A solar burst is assumed to be occurring if the radio flux increases by a factor of 200 in 500 ms at one or more frequencies, whereupon the data are stored with a time resolution of up to 1 ms. The end of the burst is determined when the real-time flux drops below the average flux over this 500 ms period. Finally, the data observed every 100 ms during the quiet Sun are averaged and stored, thereby reducing the amount of data storage. The method has been commissioned at the Chashan Solar Observatory in China: during solar flares and coronal mass ejections, the fine structure in the corresponding radio spectra is recorded with high time resolution to understand better the associated dynamic processes, whereas when the Sun is quiet the data are saved at low time resolution, thereby reducing the amount of stored data by a factor of 100 (can be set as needed). All of this helps make the system suitable for long-term observation. As for (i) the time resolutions of the data stored in the solar-burst and quiet periods and (ii) the parameters used to judge the threshold of a radio burst (e.g., 500 ms, 200 times, etc.), these can be set and changed as required. The data-processing method used in this paper can act as a reference for similar high-speed big-data systems.

Analysis of synchronized storage method for multimedia key areas based on machine learning

Aiming at the problems of high energy consumption, small amount of stored data, large standard deviation of storage space, high failure rate of storage nodes and poor quality of data storage in existing multimedia key area synchronous storage methods, a machine learning based multimedia key area synchronous storage method is proposed. Using genetic algorithm to calculate the distance between data in multimedia critical area and cluster center, and redistribute cluster set, K-Mean is realized by M-R parallel computing model. Different amounts of data are allocated to the clustered sample data storage nodes to complete the synchronous storage of multimedia key area data. The experimental results show that compared with other methods, under the network regulation of 2800 m × 800 m, the storage energy consumption of the proposed method is in the range of 10 × 103NJ-1250 × 103NJ, and the storage energy consumption is low; The maximum number of data storage is 570, and the amount of stored data is large; the standard deviation of data storage space in the key area of multimedia changes within the range of 1.3–4, and the standard deviation of storage space is small. The proposed method lays a foundation for the further development of data storage technology.

Mass data storage and sharing algorithm in distributed heterogeneous environment

In the distributed heterogeneous environment, mass data is stored in the database in the form of cloud storage, in order to improve the throughput performance of mass data storage, a mass data storage and sharing algorithm in distributed heterogeneous environment is proposed based on piecewise linear fusion equilibrium scheduling. The distributed heterogeneous environment of mass data storage is constructed, and the design of storage nodes localization is optimized, association rule feature of mass data in distributed heterogeneous environment is extracted, and the fuzzy C means clustering method is used to classify the extracted features. A piecewise linear data fusion is realized in high dimension phase space to improve the performance of equalization data storage channel, distributed storage and resource sharing of massive data in the transmission link layer are obtained. The simulation results show that throughput performance is better with this method in mass data storage, the bit amount of stored data is increased, the classification accuracy of mass data in storage space is high, and it can improve the security of massive data storage.

Optimization of electronic document management systems by means of encoding and visualization of stored data in the integrated development environment

The article considers the issue of storage, cataloging and transfer of documentation within the electronic document management system (EDMS). Modern document management solutions can be considered as the intersection of traditional and digital technologies, in particular, the storage of software code on paper. The very storage of program code in paper version implies a large archive for data storage, which includes modifications when versions of the program code are updated. Graphical ways of storing large amounts of data have been known for a long time, but they usually use a long procedure for restoring the original information and exclude recovery from paper or are significantly limited in the amount of stored data. As one of the elements of the composite electronic document management system it is proposed to use the method of graphical encoding of information (QR-code generation technology), which will increase the resistance to unauthorized modification of the program code, simplify the procedure for storing, identifying and verifying information. As an elementary block of information, it is intended to use a separate software module responsible for one of the subprograms in the RTOS environment. The proposed algorithm for storing and transferring information implies its compression by software, encryption and transmission. The article compares various compression algorithms and their efficiency. The presence of a cataloguer in each graphic element allows for quick search, comparison and verification of blocks.

Computational SS and conversion protocols in both active and passive settings

Secret sharing (SS) has been extensively studied as both a means of secure data storage and a fundamental building block for multiparty computation (MPC). For these purposes, code-efficiency and MPC-suitability are required for SS but they are incomparable. Recently, a computational SS and a conversion protocol were proposed. The computational SS is code-efficient and the conversion protocol converts shares of the computational (code-efficient) SS into those of an MPC-suitable SS, and it can be applied to reduce the amount of data storage while maintaining extendibility to MPC. However, this protocol is one-way: one cannot convert the share of MPC output value. In addition, it is only passively secure. The authors propose three protocols and a new computational SS. The first protocol is the inverse of the existing protocol, that is, it converts an MPC-suitable SS to the existing SS. The other two protocols are actively secure conversion protocols that convert shares between the new SS and an MPC-suitable SS. The new computational SS is code-efficient when the number of parties is small, so these two protocols are for converting between the code-efficient SS and an MPC-suitable SS. These two conversion protocols are actively secure in the honest majority.