Reduce Storage Requirements Research Articles

Dimensionality-reduced subspace clustering

Subspace clustering refers to the problem of clustering unlabeled high-dimensional data points into a union of low-dimensional linear subspaces, whose number, orientations, and dimensions are all unknown. In practice one may have access to dimensionality-reduced observations of the data only, resulting, e.g., from undersampling due to complexity and speed constraints on the acquisition device or mechanism. More pertinently, even if the high-dimensional data set is available it is often desirable to rst project the data points into a lower-dimensional space and to perform clustering there; this reduces storage requirements and computational cost. The purpose of this paper is to quantify the impact of dimensionality reduction through random projection on the performance of three subspace clustering algorithms, all of which are based on principles from sparse signal recovery. Specically, we analyze the thresholding based subspace clustering (TSC) algorithm, the sparse subspace clustering (SSC) algorithm, and an orthogonal matching pursuit variant thereof (SSC-OMP). We nd, for all three algorithms, that dimensionality reduction down to the order of the subspace dimensions is possible without incurring signicant performance degradation. Moreover, these results are order-wise optimal in the sense that reducing the dimensionality further leads to a fundamentally ill-posed clustering problem. Our ndings carry over to the noisy case as illustrated through analytical results for TSC and simulations for SSC and SSC-OMP. Extensive experiments on synthetic and real data complement our theoretical ndings.

IoT-Lite: a lightweight semantic model for the internet of things and its use with dynamic semantics

Over the past few years, the semantics community has developed several ontologies to describe concepts and relationships for internet of things (IoT) applications. A key problem is that most of the IoT-related semantic descriptions are not as widely adopted as expected. One of the main concerns of users and developers is that semantic techniques increase the complexity and processing time, and therefore, they are unsuitable for dynamic and responsive environments such as the IoT. To address this concern, we propose IoT-Lite, an instantiation of the semantic sensor network ontology to describe key IoT concepts allowing interoperability and discovery of sensory data in heterogeneous IoT platforms by a lightweight semantics. We propose 10 rules for good and scalable semantic model design and follow them to create IoT-Lite. We also demonstrate the scalability of IoT-Lite by providing some experimental analysis and assess IoT-Lite against another solution in terms of round trip time performance for query-response times. We have linked IoT-Lite with stream annotation ontology, to allow queries over stream data annotations, and we have also added dynamic semantics in the form of MathML annotations to IoT-Lite. Dynamic semantics allows the annotation of spatio-temporal values, reducing storage requirements and therefore the response time for queries. Dynamic semantics stores mathematical formulas to recover estimated values when actual values are missing.

Image Compression Based on Multilevel Adaptive Thresholding using Meta-Data Heuristics

Satellite image processing involves very often the need of compression. The compression of satellite images will reduce storage requirements and conserves transmission bandwidth. In this paper, a lossy image compression method is proposed based on multilevel adaptive thresholding using Meta-Data heuristics to compress the Landsat-8 satellite images. In the proposed method the number of thresholds is fixed in accordance with the bitrate required and the Peak Signal to Noise Ratio (PSNR) is improved by entropy based adaptive thresholding. Test image of Landsat-8, Band 3, 5 is used for performing the compression and the performance metric PSNR is measured for uniform thresholding and the proposed method. The proposed method gives improvement in the PSNR and the method is computationally simulated using Fixed Point Binary scaling.

Efficient interpolation of precomputed kinetic data employing reduced multivariate Hermite Splines

Abstract Detailed surface kinetics can be efficiently implemented into complex reactor simulations by the use of precomputed solutions of the chemical rate equations. In this work we propose reduced Hermite splines as an interpolation method for the precomputed rate data. The reduced Hermite splines require significantly less storage space and less execution time than the tensor product splines that have been previously used for this task. Using previously published test cases on NH 3 oxidation, we demonstrate that despite of the reduced storage requirements and faster interpolation times, reduced Hermite interpolation achieves the same interpolation accuracy as conventional full tensor product splines. In many cases, the derivatives of the interpolated function with respect to the input quantities can be cheaply obtained during the computation of the function values. If these derivatives are provided as additional information, the accuracy of the reduced Hermite method even outperforms conventional tensor product splines, in our demonstration example by a factor of three. The advantage of the reduced Hermite schemes with respect to storage requirements and evaluation time strongly increase with the dimensionality of the interpolation problem. It is therefore expected that the reduced Hermite splines will allow to significantly extend the application range of solution mapping methods to higher dimensional problems (i.e. problems with a larger number of relevant chemical species).

Generating a Corpus of Mobile Forensic Images for Masquerading user Experimentation.

The Periodic Mobile Forensics (PMF) system investigates user behavior on mobile devices. It applies forensic techniques to an enterprise mobile infrastructure, utilizing an on-device agent named TractorBeam. The agent collects changed storage locations for later acquisition, reconstruction, and analysis. TractorBeam provides its data to an enterprise infrastructure that consists of a cloud-based queuing service, relational database, and analytical framework for running forensic processes. During a 3-month experiment with Purdue University, TractorBeam was utilized in a simulated operational setting across 34 users to evaluate techniques to identify masquerading users (i.e., users other than the intended device user). The research team surmises that all masqueraders are undesirable to an enterprise, even when a masquerader lacks malicious intent. The PMF system reconstructed 821 forensic images, extracted one million audit events, and accurately detected masqueraders. Evaluation revealed that developed methods reduced storage requirements 50-fold. This paper describes the PMF architecture, performance of TractorBeam throughout the protocol, and results of the masquerading user analysis.

Wavenumber–frequency deconvolution of aeroacoustic microphone phased array data of arbitrary coherence

Deconvolution of aeroacoustic data acquired with microphone phased arrays is a computationally challenging task for distributed sources with arbitrary coherence. A new technique for performing such deconvolution is proposed. This technique relies on analysis of the array data in the wavenumber–frequency domain, allowing for fast convolution and reduced storage requirements when compared to traditional coherent deconvolution. A positive semidefinite constraint for the iterative deconvolution procedure is implemented and shows improved behavior in terms of quantifiable convergence metrics when compared to a standalone covariance inequality constraint. A series of simulations validates the method׳s ability to resolve coherence and phase angle relationships between partially coherent sources, as well as determines convergence criteria for deconvolution analysis. Simulations for point sources near the microphone phased array show potential for handling such data in the wavenumber–frequency domain. In particular, a physics-based integration boundary calculation is described, and can successfully isolate sources and track the appropriate integration bounds with and without the presence of flow. Magnitude and phase relationships between multiple sources are successfully extracted. Limitations of the deconvolution technique are determined from the simulations, particularly in the context of a simulated acoustic field in a closed test section wind tunnel with strong boundary layer contamination. A final application to a trailing edge noise experiment conducted in an open-jet wind tunnel matches best estimates of acoustic levels from traditional calculation methods and qualitatively assesses the coherence characteristics of the trailing edge noise source.

Temporal and Spatial Interpolation Errors of High-Resolution Modeled Atmospheric Fields

AbstractGeneral circulation models can now be run at very high spatial resolutions to capture finescale features, but saving the full-spatial-resolution output at every model time step is usually not practical because of storage limitations. To reduce storage requirements, the model output may be produced at reduced temporal and/or spatial resolutions. When this reduced-resolution output is then used in situations where spatiotemporal interpolation is required, such as the generation of synthetic observations for observing system simulation experiments, interpolation errors can significantly affect the quality and usefulness of the reduced-resolution model output. Although it is common in practice to record model output at the highest possible spatial resolution with relatively infrequent temporal output, this may not be the best option to minimize interpolation errors. In this study, two examples using a high-resolution global run of the Goddard Earth Observing System Model, version 5 (GEOS-5), are presented to illustrate cases in which the optimal output dataset configurations for interpolation have high temporal frequency but reduced spatial resolutions. Interpolation errors of tropospheric temperature, specific humidity, and wind fields are investigated. The relationship between spatial and temporal output resolutions and interpolation errors is also characterized for the example model.

Schedule Coordination, Delay Propagation, and Disruption Resilience in Intermodal Logistics Networks

Three optimization models were developed to manage operations in intermodal logistics networks, from routine scheduling delay to recovery from major disruptions. Efficient transfer coordination in intermodal logistics networks can reduce the freight dwell time at transfer terminals where various routes interconnect and can reduce storage requirements at terminals. The first model coordinated vehicle schedules and cargo transfers at intermodal freight terminals, primarily by optimizing coordinated service frequencies and slack times. When delay perturbations propagated within a logistics network that used schedule coordination, a second model determined whether each ready outbound vehicle should have been dispatched immediately or held to wait for late incoming vehicles. Finally, a set of optimal resilient actions was considered by the proposed third model during the postdisruption phases, such as switching shipping modes and routes, renting other carriers' capacities, reallocating local trucks, and prioritizing the order of shipments because of limited capacities.

High Quality Video Compression for Noise Removal Classification using Discrete Cosine Transforms based Spatial Detection

To present an inventive method for lossless solidity of discrete-color video frame, such as map video frame, graphics, gif, as well as binary video frame. In lossless density techniques, the original video frame can be perfectly improved from the compacted video frame. These are also called low since they do not include noise to the signal.it is also known as entropy coding since it use transformation technique to eliminate/minimize redundancy. An efficient prediction-based lossless compression scheme for DCT edge-preserving Maximum Homogeneity Neighbors Filter compression technique proposed to size the image. The purpose of the lossless video frame compression technique is to reduce the amount of data required to represent the video frame with less deprivation in the visual value and without any in order loss. In a monochrome video frame, the neighboring pixels are more interrelated. Lossless compression is used only for a few applications with stringent requirements such as medical imaging. It not only reduces storage requirements but also overall execution time. The probability was obtained from a very large set of separate color video frames which are also used for arithmetic coding. The second component is the row-column reduction coding, which will encode those block that are not in the codebook. The purpose of the lossless video frame compression technique is to reduce the amount of data required to represents the video frame with less degradation in the visual quality and without any information loss. In a monochrome video frame, the neighboring pixels are more correlated. The discrete cosine transform based filters are commonly used to reduce the job loss between the pixels and for power compaction.

Towards a unified multiresolution vision model for autonomous ground robots

While remotely operated unmanned vehicles are increasingly a part of everyday life, truly autonomous robots capable of independent operation in dynamic environments have yet to be realized — particularly in the case of ground robots required to interact with humans and their environment. We present a unified multiresolution vision model for this application designed to provide the wide field of view required to maintain situational awareness and sufficient visual acuity to recognize elements of the environment while permitting feasible implementations in real-time vision applications. The model features a kind of color-constant processing through single-opponent color channels and contrast invariant oriented edge detection using a novel implementation of the Combination of Receptive Fields model. The model provides color and edge-based salience assessment, as well as a compressed color image representation suitable for subsequent object identification. We show that bottom-up visual saliency computed using this model is competitive with the current state-of-the-art while allowing computation in a compressed domain and mimicking the human visual system with nearly half (45%) of computational effort focused within the fovea. This method reduces storage requirement of the image pyramid to less than 5% of the full image, and computation in this domain reduces model complexity in terms of both computational costs and memory requirements accordingly. We also quantitatively evaluate the model for its application domain by using it with a camera/lens system with a 185°field of view capturing 3.5M pixel color images by using a tuned salience model to predict human fixations.

Analysis of the Jacobian-free multiscale method (JFMM)

In this paper we perform in-depth analysis of Jacobian-free multiscale method (JFMM) in which explicit computation of the Jacobian matrix at the macroscale is circumvented using a Newton---Krylov process. Not having to explicitly compute and store the Jacobian matrix at each Newton step reduces storage requirements and computational costs compared to previous efforts based on homogenized material coefficients with Jacobian computation at every Newton step. We present an estimate of the optimal perturbation step-size that minimizes the finite difference approximation error associated with the Jacobian-vector product in the Jacobian-free approach. Two- and three-dimensional numerical examples demonstrate that while the rate of convergence of Newton iterations for the JFMM and the computational homogenization-based two-level finite element $$(\hbox {FE}^{2})$$(FE2) multiscale method is comparable, the computational cost of JFMM varies linearly with increasing number of degrees of freedom (n) at the macroscale, and not exponentially as in the $$\hbox {FE}^{2}$$FE2 method. The storage requirement for the method increases linearly with increasing n at the macroscale, whereas, it increases as approximately $$O(n^{8/5})$$O(n8/5) and $$O(n^{9/5})$$O(n9/5) for the $$\hbox {FE}^{2}$$FE2 method in two- and three-dimensions, respectively.

Complex Luminaires

Simulating a complex luminaire such as a chandelier is expensive and slow, even using state-of-the-art algorithms. A more practical alternative is to use precomputation to accelerate rendering. Prior approaches cached information on an aperture surface that separates the luminaire from the scene, but many luminaires have large or ill-defined apertures leading to excessive data storage and inaccurate results. In this article, we separate luminaire rendering into illumination and appearance components. A precomputation stage simulates the complex light flow inside the luminaire to generate two data structures: a set of anisotropic point lights (APLs) and a radiance volume. The APLs are located near apparent sources and represent the light leaving the luminaire, allowing its nearand far-field illumination to be accurately and efficiently computed at render time. The luminaire's appearance consists of high- and low-frequency components, which are both visually important. High-frequency components are computed dynamically at render time, while the more computationally expensive low-frequency components are approximated using the precomputed radiance volume. Results are shown for several complex luminaires, demonstrating orders of magnitude faster rendering compared to the best global illumination algorithms and higher fidelity with greatly reduced storage requirements compared to previous precomputed approaches.

Learning predictable binary codes for face indexing

High dimensional dense features have been shown to be useful for face recognition, but result in high query time when searching a large-scale face database. Hence binary codes are often used to obtain fast query speeds as well as reduce storage requirements. However, binary codes for face features can become unstable and unpredictable due to face variations induced by pose, expression and illumination. This paper proposes a predictable hash code algorithm to map face samples in the original feature space to Hamming space. First, we discuss the ‘predictability’ of hash codes for face indexing. Second, we formulate the predictable hash coding problem as a non-convex combinatorial optimization problem, in which the distance between codes for samples from the same class is minimized while the distance between codes for samples from different classes is maximized. An Expectation Maximization method is introduced to iteratively find a sparse and predictable linear mapping. Lastly, a deep feature representation is learned to further enhance the predictability of binary codes. Experimental results on three commonly used face databases demonstrate the superiority of our predictable hash coding algorithm on large-scale problems.

An efficient data preprocessing approach for large scale medical data mining.

The size of medical datasets is usually very large, which directly affects the computational cost of the data mining process. Instance selection is a data preprocessing step in the knowledge discovery process, which can be employed to reduce storage requirements while also maintaining the mining quality. This process aims to filter out outliers (or noisy data) from a given (training) dataset. However, when the dataset is very large in size, more time is required to accomplish the instance selection task. In this paper, we introduce an efficient data preprocessing approach (EDP), which is composed of two steps. The first step is based on training a model over a small amount of training data after preforming instance selection. The model is then used to identify the rest of the large amount of training data. Experiments are conducted based on two medical datasets for breast cancer and protein homology prediction problems that contain over 100000 data samples. In addition, three well-known instance selection algorithms are used, IB3, DROP3, and genetic algorithms. On the other hand, three popular classification techniques are used to construct the learning models for comparison, namely the CART decision tree, k-nearest neighbor (k-NN), and support vector machine (SVM). The results show that our proposed approach not only reduces the computational cost by nearly a factor of two or three over three other state-of-the-art algorithms, but also maintains the final classification accuracy. To perform instance selection over large scale medical datasets, it requires a large computational cost to directly execute existing instance selection algorithms. Our proposed EDP approach solves this problem by training a learning model to recognize good and noisy data. To consider both computational complexity and final classification accuracy, the proposed EDP has been demonstrated its efficiency and effectiveness in the large scale instance selection problem.

An improved fast local level set method for three-dimensional inverse gravimetry

We propose an improved fast local level set method for the inverse problem ofgravimetry by developing two novel algorithms: one is of linear complexitydesigned for computing the Frechet derivative of the nonlinear domain inverseproblem, and the other is designed for carrying out numerical continuationrapidly so as to obtain fictitious full measurement data from partialmeasurement. Since the Laplacian kernel is symmetric and translationally invariant, wedesign certain affine transformations to speed up the computational process inevaluating the Frechet derivative; since it decays rapidly away from diagonal,we carry out low-rank matrix approximation to reduce storage requirements. Theseproperties are eventually translated into an algorithm of linear complexity andlinear storage requirement for computing the derivative. Since the single layerdensity function, used in representing the potential, is smooth and periodic onan artificial hypersurface enclosing the target domain, the spectral expansionis allowed to approximate this density function, which eventually leads to rapidalgorithms for carrying out the numerical continuation in both 2-D and 3-Dcases. 2-D and 3-D numerical examples illustrate that this improved level-setmethod is capable of computing high-resolution inversions and handling 3-Dlarge-scale inverse gravimetry problems.

Adjoint‐based error estimation and mesh adaptation for hybridized discontinuous Galerkin methods

SummaryWe present a robust and efficient target‐based mesh adaptation methodology, building on hybridized discontinuous Galerkin schemes for (nonlinear) convection–diffusion problems, including the compressible Euler and Navier–Stokes equations. The hybridization of finite element discretizations has the main advantage that the resulting set of algebraic equations has globally coupled degrees of freedom (DOFs) only on the skeleton of the computational mesh. Consequently, solving for these DOFs involves the solution of a potentially much smaller system. This not only reduces storage requirements but also allows for a faster solution with iterative solvers. The mesh adaptation is driven by an error estimate obtained via a discrete adjoint approach. Furthermore, the computed target functional can be corrected with this error estimate to obtain an even more accurate value. The aim of this paper is twofold: Firstly, to show the superiority of adjoint‐based mesh adaptation over uniform and residual‐based mesh refinement and secondly, to investigate the efficiency of the global error estimate. Copyright © 2014 John Wiley & Sons, Ltd.

Local Painting and Deformation of Meshes on the GPU

AbstractWe present a novel method to adaptively apply modifications to scene data stored in GPU memory. Such modifications may include interactive painting and sculpting operations in an authoring tool, or deformations resulting from collisions between scene objects detected by a physics engine. We only allocate GPU memory for the faces affected by these modifications to store fine‐scale colour or displacement values. This requires dynamic GPU memory management in order to assign and adaptively apply edits to individual faces at runtime. We present such a memory management technique based on a scan‐operation that is efficiently parallelizable. Since our approach runs entirely on the GPU, we avoid costly CPU–GPU memory transfer and eliminate typical bandwidth limitations. This minimizes runtime overhead to under a millisecond and makes our method ideally suited to many real‐time applications such as video games and interactive authoring tools. In addition, our algorithm significantly reduces storage requirements and allows for much higher resolution content compared to traditional global texturing approaches. Our technique can be applied to various mesh representations, including Catmull–Clark subdivision surfaces, as well as standard triangle and quad meshes. In this paper, we demonstrate several scenarios for these mesh types where our algorithm enables adaptive mesh refinement, local surface deformations and interactive on‐mesh painting and sculpting.

Cost analysis of concepts for a demand oriented biogas supply for flexible power generation

With the share of intermittent renewable energies within the electricity system rising, balancing services from dispatchable power plants are of increasing importance. Highlighting the importance of the need to keeping fuel costs for flexible power generation to a minimum, the study aims to identify favourable biogas plant configurations, supplying biogas on demand. A cost analysis of five configurations based on biogas storing and flexible biogas production concepts has been carried out. Results show that additional flexibility costs for a biogas supply of 8h per day range between 2€ and 11€MWh−1 and for a 72h period without biogas demand from 9€ to 19€MWh−1. While biogas storage concepts were identified as favourable short term supply configurations, flexible biogas production concepts profit from reduced storage requirements at plants with large biogas production capacities or for periods of several hours without biogas demand.

Effect of Hybrid Data Compression Technique on the Diagnostic Accuracy of Region of Interest (ROI) on MRI Image of a Spine Disc Prolapse

The cost of transmitting and archiving medical images are quite prohibitive due to their large sizes especially in digital radiology system such as: picture archiving and communication systems and teleradiology. In order to reduce storage requirements and improve transmission rate, there is need for compression. Usually, radiologists are only interested in the abnormal region of the image (known as the region of interest) in making diagnosis and interpretations; hence, this work investigates the effect of hybrid data compression technique on the diagnostic accuracy of region of interest (ROI) on magnetic resonance image (MRI) of a spine disc prolapsed. We extract the ROI from the original image and apply lossless Wavelet-Based Compression (WBC) on the ROI while the remainder image known as the non-region of interest is compressed using discrete cosine transform (DCT). A compression ratio of 7:1 was achieved. Finally, the diagnostic accuracy of the compressed ROI image was evaluated subjectively by a group of 30 evaluators comprising of 20 radiologists and 10 Radiographers. The results obtained show a 100% acceptance of the compressed ROI for healthy diagnosis and interpretation. General Terms Data compression technique

A comparison of hybridized and standard DG methods for target-based hp-adaptive simulation of compressible flow

Objective: We present a comparison between hybridized and non-hybridized discontinuous Galerkin methods in the context of target-based hp-adaptation for compressible flow problems. The aim is to provide a critical assessment of the computational efficiency of hybridized DG methods.Method: Hybridization of finite element discretizations has the main advantage, that the resulting set of algebraic equations has globally coupled degrees of freedom only on the skeleton of the computational mesh. Consequently, solving for these degrees of freedom involves the solution of a potentially much smaller system. This not only reduces storage requirements, but also allows for a faster solution with iterative solvers. Using a discrete-adjoint approach, sensitivities with respect to output functionals are computed to drive the adaptation. From the error distribution given by the adjoint-based error estimator, h- or p-refinement is chosen based on the smoothness of the solution which can be quantified by properly-chosen smoothness indicators.Results: Numerical results are shown for subsonic, transonic, and supersonic flow around the NACA0012 airfoil. $hp$-adaptation proves to be superior to pure h-adaptation if discontinuous or singular flow features are involved. In all cases, a higher polynomial degree turns out to be beneficial. We show that for polynomial degree of approximation p=2 and higher, and for a broad range of test cases, HDG performs better than DG in terms of runtime and memory requirements.