Reduce Memory Requirements Research Articles

TempoGAN

We propose a temporally coherent generative model addressing the super-resolution problem for fluid flows. Our work represents a first approach to synthesize four-dimensional physics fields with neural networks. Based on a conditional generative adversarial network that is designed for the inference of three-dimensional volumetric data, our model generates consistent and detailed results by using a novel temporal discriminator, in addition to the commonly used spatial one. Our experiments show that the generator is able to infer more realistic high-resolution details by using additional physical quantities, such as low-resolution velocities or vorticities. Besides improvements in the training process and in the generated outputs, these inputs offer means for artistic control as well. We additionally employ a physics-aware data augmentation step, which is crucial to avoid overfitting and to reduce memory requirements. In this way, our network learns to generate adverted quantities with highly detailed, realistic, and temporally coherent features. Our method works instantaneously, using only a single time-step of low-resolution fluid data. We demonstrate the abilities of our method using a variety of complex inputs and applications in two and three dimensions.

NullHop: A Flexible Convolutional Neural Network Accelerator Based on Sparse Representations of Feature Maps.

Convolutional neural networks (CNNs) have become the dominant neural network architecture for solving many state-of-the-art (SOA) visual processing tasks. Even though graphical processing units are most often used in training and deploying CNNs, their power efficiency is less than 10 GOp/s/W for single-frame runtime inference. We propose a flexible and efficient CNN accelerator architecture called NullHop that implements SOA CNNs useful for low-power and low-latency application scenarios. NullHop exploits the sparsity of neuron activations in CNNs to accelerate the computation and reduce memory requirements. The flexible architecture allows high utilization of available computing resources across kernel sizes ranging from 1×1 to 7×7 . NullHop can process up to 128 input and 128 output feature maps per layer in a single pass. We implemented the proposed architecture on a Xilinx Zynq field-programmable gate array (FPGA) platform and presented the results showing how our implementation reduces external memory transfers and compute time in five different CNNs ranging from small ones up to the widely known large VGG16 and VGG19 CNNs. Postsynthesis simulations using Mentor Modelsim in a 28-nm process with a clock frequency of 500 MHz show that the VGG19 network achieves over 450 GOp/s. By exploiting sparsity, NullHop achieves an efficiency of 368%, maintains over 98% utilization of the multiply-accumulate units, and achieves a power efficiency of over 3 TOp/s/W in a core area of 6.3 mm2. As further proof of NullHop's usability, we interfaced its FPGA implementation with a neuromorphic event camera for real-time interactive demonstrations.

Dynamic compression schemes for graph coloring.

MotivationTechnological advancements in high-throughput DNA sequencing have led to an exponential growth of sequencing data being produced and stored as a byproduct of biomedical research. Despite its public availability, a majority of this data remains hard to query for the research community due to a lack of efficient data representation and indexing solutions. One of the available techniques to represent read data is a condensed form as an assembly graph. Such a representation contains all sequence information but does not store contextual information and metadata.ResultsWe present two new approaches for a compressed representation of a graph coloring: a lossless compression scheme based on a novel application of wavelet tries as well as a highly accurate lossy compression based on a set of Bloom filters. Both strategies retain a coloring even when adding to the underlying graph topology. We present construction and merge procedures for both methods and evaluate their performance on a wide range of different datasets. By dropping the requirement of a fully lossless compression and using the topological information of the underlying graph, we can reduce memory requirements by up to three orders of magnitude. Representing individual colors as independently stored modules, our approaches can be efficiently parallelized and provide strategies for dynamic use. These properties allow for an easy upscaling to the problem sizes common to the biomedical domain.Availability and implementationWe provide prototype implementations in C++, summaries of our experiments as well as links to all datasets publicly at https://github.com/ratschlab/graph_annotation.Supplementary information Supplementary data are available at Bioinformatics online.

Quality Robust Mixtures of Deep Neural Networks.

We study deep neural networks for classification of images with quality distortions. Deep network performance on poor quality images can be greatly improved if the network is fine-tuned with distorted data. However, it is difficult for a single fine-tuned network to perform well across multiple distortion types. We propose a mixture of experts based ensemble method, MixQualNet, that is robust to multiple different types of distortions. The "experts" in our model are trained on a particular type of distortion. The output of the model is a weighted sum of the expert models, where the weights are determined by a separate gating network. The gating network is trained to predict weights for a particular distortion type and level. During testing, the network is blind to the distortion level and type, yet can still assign appropriate weights to the expert models. In order to reduce the computational complexity, we introduce weight sharing into the MixQualNet. We utilize the TreeNet weight sharing architecture as well as introduce the Inverted TreeNet architecture. While both weight sharing architectures reduce memory requirements, our proposed Inverted TreeNet also achieves improved accuracy.

Bayesian Conditional Density Filtering

ABSTRACTWe propose a conditional density filtering (C-DF) algorithm for efficient online Bayesian inference. C-DF adapts MCMC sampling to the online setting, sampling from approximations to conditional posterior distributions obtained by propagating surrogate conditional sufficient statistics (a function of data and parameter estimates) as new data arrive. These quantities eliminate the need to store or process the entire dataset simultaneously and offer a number of desirable features. Often, these include a reduction in memory requirements and runtime and improved mixing, along with state-of-the-art parameter inference and prediction. These improvements are demonstrated through several illustrative examples including an application to high dimensional compressed regression. In the cases where dimension of the model parameter does not grow with time, we also establish sufficient conditions under which C-DF samples converge to the target posterior distribution asymptotically as sampling proceeds and more data arrive. Supplementary materials of C-DF are available online.

Modeling Nonlinear Synaptic Dynamics: A Laguerre-Volterra Network Framework for Improved Computational Efficiency in Large Scale Simulations.

Synapses are key components in signal transmission in the brain, often exhibiting complex non-linear dynamics. Yet, they are often crudely modelled as linear exponential equations in large-scale neuron network simulations. Mechanistic models that use detailed channel receptor kinetics more closely replicate the nonlinear dynamics observed at synapses, but use of such models are generally restricted to small scale simulations due to their computational complexity. Previously, we have developed an ``input-output'' (IO) synapse model using the Volterra functional series to estimate nonlinear synaptic dynamics. Here, we present an improvement on the IO synapse model using the extbf{Laguerre-Volterra network (LVN) framework. We demonstrate that utilization of the LVN framework helps reduce memory requirements and improves the simulation speed in comparison to the previous iteration of the IO synapse model. We present results that demonstrate the accuracy, memory efficiency, and speed of the LVN model that can be extended to simulations with large numbers of synapses. Our efforts enable complex nonlinear synaptic dynamics to be modelled in large-scale network models, allowing us to explore how synaptic activity may influence network behavior and affects memory, learning, and neurodegenerative diseases.

Deep Classification of Epileptic Signals.

Electrophysiological observation plays a major role in epilepsy evaluation. However, human interpretation of brain signals is subjective and prone to misdiagnosis. Automating this process, especially seizure detection relying on scalpbased Electroencephalography (EEG) and intracranial EEG, has been the focus of research over recent decades. Nevertheless, its numerous challenges have inhibited a definitive solution. Inspired by recent advances in deep learning, here we describe a new classification approach for EEG time series based on Recurrent Neural Networks (RNNs) via the use of Long- Short Term Memory (LSTM) networks. The proposed deep network effectively learns and models discriminative temporal patterns from EEG sequential data. Especially, the features are automatically discovered from the raw EEG data without any pre-processing step, eliminating humans from laborious feature design task. Our light-weight system has a low computational complexity and reduced memory requirement for large training datasets. On a public dataset, a multi-fold cross-validation scheme of the proposed architecture exhibited an average validation accuracy of 95.54% and an average AUC of 0.9582 of the ROC curve among all sets defined in the experiment. This work reinforces the benefits of deep learning to be further attended in clinical applications and neuroscientific research.

A Fast IE-FFT Algorithm to Analyze Electrically Large Planar Microstrip Antenna Arrays

A fast integral-equation fast Fourier transform (IE-FFT) algorithm is developed, analyzed, and applied to the electromagnetic solution of radiation problems. The solution begins by the method-of-moments solution of a mixed potential integral equation on electrically large antenna arrays. Similar to other grid-based algorithms, the IE-FFT uses a Cartesian grid to significantly reduce memory requirement and speed up CPU time for both matrix assembly and matrix-vector multiplication when used with an iterative solver. The IE-FFT algorithm employs two discretizations, one for unknown current on an unstructured triangular mesh and the other on a uniform Cartesian grid for interpolating the layered medium's Green's functions. The uniform interpolation of the Green's functions allows us a global FFT for well-separated far-interaction terms. However, the near-interaction terms should be effectively corrected. A simple and efficient Lagrangian interpolation of the Green's functions is proposed. For a 2.5-D planar layered structure problem, the complexity of an IE-FFT is found to approximately $O(N)$ and $O({N\,{\rm{log}}\,N})$ for memory and CPU time, respectively.

TernaryNet: faster deep model inference without GPUs for medical 3D segmentation using sparse and binary convolutions.

Deep convolutional neural networks (DCNN) are currently ubiquitous in medical imaging. While their versatility and high-quality results for common image analysis tasks including segmentation, localisation and prediction is astonishing, the large representational power comes at the cost of highly demanding computational effort. This limits their practical applications for image-guided interventions and diagnostic (point-of-care) support using mobile devices without graphics processing units (GPU). We propose a new scheme that approximates both trainable weights and neural activations in deep networks by ternary values and tackles the open question of backpropagation when dealing with non-differentiable functions. Our solution enables the removal of the expensive floating-point matrix multiplications throughout any convolutional neural network and replaces them by energy- and time-preserving binary operators and population counts. We evaluate our approach for the segmentation of the pancreas in CT. Here, our ternary approximation within a fully convolutional network leads to more than 90% memory reductions and high accuracy (without any post-processing) with a Dice overlap of 71.0% that comes close to the one obtained when using networks with high-precision weights and activations. We further provide a concept for sub-second inference without GPUs and demonstrate significant improvements in comparison with binary quantisation and without our proposed ternary hyperbolic tangent continuation. We present a key enabling technique for highly efficient DCNN inference without GPUs that will help to bring the advances of deep learning to practical clinical applications. It has also great promise for improving accuracies in large-scale medical data retrieval.

SupAUTH: A new approach to supply chain authentication for the IoT

AbstractRecent advances of the Internet of Things (IoT) technologies have enhanced the use of radio‐frequency identification‐based tracking system to be widely deployed in supply chain management covering every step involved in the flow of merchandise from the supplier to the customer to ensure a trustworthy delivery environment. Such authentication system (also known as path authentication) not only guarantees the merchandise to be available in the right destination with no discrepancies and errors but also ensures the route of the merchandise progress to be valid. This paper outlines the current state‐of‐the‐art cryptographic solutions for path authentication, highlights their properties and weakness, and proposes a novel, privacy‐preserving, and efficient solution. Compared with the existing elliptic curve ElGamal re‐encryption–based solution, our homomorphic message authentication code on arithmetic circuit–based solution offers less memory storage (with limited scalability) and no computational requirement on the reader. Moreover, we allow computational ability inside the tag that articulates a new privacy direction to the state‐of‐the‐art path privacy. This privacy notion helps support the confidentiality of the tag movement in the context of IoT‐enabled cross‐organizational tracking environment where the stakeholders can be from different organizations associated together with the merchandise being delivered. As a potential extension to the path authentication protocol, we further propose a polynomial‐based mutual authentication as a security extension and batch initialization as an efficiency extension. Besides our brief security and privacy analysis, our evaluation shows that the proposed solution can significantly reduce memory requirements on tags with marginal computational overhead to ensure transmission path confidentiality. We observe that SupAUTH requires maximum 513‐bit tag memory and 57.3 ms of processing time during evaluation, which is not only practical but also suitable for any suitable low‐cost radio‐frequency identification deployment in IoT.

A novel efficient inversion method for three-dimensional NMR and the optimization of activation sequences and acquisition parameters

A three-dimensional (3D) nuclear magnetic resonance (NMR) spectrum can simultaneously provide distributions of longitudinal relaxation time (T1), transverse relaxation time (T2), and diffusivity (D); thus, it greatly improves the capacity of fluid identification, typing, and quantitative evaluations. However, several challenges that significantly hinder the widespread application of this technique remain. The primary challenges are the high time and memory costs associated with the current 3D NMR inversion algorithms. In addition, an activation sequence optimization method for 3D NMR inversions has not been developed. In this paper, a novel inversion method for 3D NMR spectra and a detailed optimization method for activation sequences and acquisition parameters were proposed. The novel method, namely randomized singular value decomposition (RSVD) inversion algorithm, can reduce memory requirements and ensure computational efficiency and accuracy. Window averaging (WA) technique was also adopted in this study to further increase computational speed. The optimized method for pulse sequences is mainly based on projections of the 3D NMR spectra in the two-dimensional (2D) and one-dimensional (1D) domains. These projections can identify missing NMR properties of different fluids. Because of the efficiency and stability of this novel algorithm and the optimized strategy, the proposed methods presented in this paper could further promote the widespread application of 3D NMR.

A Practical Randomized CP Tensor Decomposition

The CANDECOMP/PARAFAC (CP) decomposition is a leading method for the analysis of multiway data. The standard alternating least squares algorithm for the CP decomposition (CP-ALS) involves a series of highly overdetermined linear least squares problems. We extend randomized least squares methods to tensors and show the workload of CP-ALS can be drastically reduced without a sacrifice in quality. We introduce techniques for efficiently preprocessing, sampling, and computing randomized least squares on a dense tensor of arbitrary order, as well as an efficient sampling-based technique for checking the stopping condition. We also show more generally that the Khatri-Rao product (used within the CP-ALS iteration) produces conditions favorable for direct sampling. In numerical results, we see improvements in speed, reductions in memory requirements, and robustness with respect to initialization.

Recursive DBPSO for Computationally Efficient Electronic Nose System

Feature-rank-code-based classifiers have been proposed recently in order to reduce the complexity for electronic nose system (ENS). The performance of these classifiers is degraded when the discriminatory information of gases lie in the actual feature values not in the ranks of features. To overcome the problem, a gas identification system based on a simple distance measure in combination with different type of features is proposed. In order to improve the computational cost of the existing ENS, a novel combination of optimum subset of features is proposed. To achieve the aim of low computational cost, a large feature vector is recursively reduced to a small number of features without compromising the classification accuracy of the system. Discrete binary particle swarm optimization, a metaheuristic for global search is used in a recursive setup to select the optimum subset of features for classification. Euclidian distance is used as a similarity measure for identification of different industrial gases. The proposed system is tested for the identification of 13 different industrial gases, namely, C3H8, Cl2, CO, CO2, SO2, NO2, NH3, C2H4O, C3H6O, C2H4, C2H6O, C7H8, and CH4. These gases are contained in three different data sets; two among these data sets are acquired experimentally in a laboratory setup, while one of these data sets is taken from the University of California at Irvine machine learning repository. Results reveal that the computational cost and the memory requirement of an ENS can be significantly reduced by combining different type of features. An average classification accuracy of 98.17% is achieved by the proposed system with an average 69.04% reduction in memory requirement using only three to five features.

Parallel Integral Equation Based Nonoverlapping DDM for Solving Electrically-Large Scattering Problems in Half Space

In this paper, a parallel nonoverlapping non-conformal domain decomposition method using combined-field integral equation is proposed for fast and accurate analysis of electrically large objects situated in half space. This method decomposes the whole model into several easily solvable closed subdomains in a flexible way, and an explicit boundary condition is applied to ensure the continuity of electric currents across the boundary. The adaptive direction partitioning parallelization scheme for the oct-tree of a multilevel fast multipole algorithm is adopted to accelerate matrix-vector multiplications of subdomains as well as the coupling between them. The introducing of real image source to account for far interaction matrix greatly accelerates the process and reduces memory requirement with precision guaranteed. Numerical examples demonstrate that the proposed method is able to simulate realistic problems with a maximum dimension greater than 1000 wavelengths.

Dynamic computational resource management technique for solving engineering problem

Fluid dynamics and transport phenomena, such as heat and mass transfer, play a vitally important role in human life. Gases and liquids surround us, flow inside our bodies, and have a profound influence on the environment in which we live. Fluid flows produce winds, rains, floods, and hurricanes. Convection and diffusion are responsible for temperature fluctuations and transport of pollutants in air, water or soil. The ability to understand, predict, and control transport phenomena is essential for many industrial applications, such as aerodynamic shape design, oil recovery from an underground reservoir, or multiphase/multi-component flows in furnaces, heat ex-changers, and chemical reactors. This ability offers substantial economic benefits and contributes to human well-being. Heating, air conditioning, and weather fore-cast have become an integral part of our everyday life. The advances in numerical methods have enabled the development of a new approach in terms of solving engineering problems. Often it has been observed that these methods have resulted in demand for higher computational requirements. In current project we aim to model convective heat transfer in a channel filled with water. It also demonstrates a technique to reduce memory requirement, by solving the model repeatedly on a pseudo-periodic section of the channel. Each solution corresponds to a different section, and before each solution step the temperature of the outlet boundary from the previous solution is mapped to the inlet boundary

A Low-Rank Multigrid Method for the Stochastic Steady-State Diffusion Problem

We study a multigrid method for solving large linear systems of equations with tensor product structure. Such systems are obtained from stochastic finite element discretization of stochastic partial differential equations such as the steady-state diffusion problem with random coefficients. When the variance in the problem is not too large, the solution can be well approximated by a low-rank object. In the proposed multigrid algorithm, the matrix iterates are truncated to low rank to reduce memory requirements and computational effort. The method is proved convergent with an analytic error bound. Numerical experiments show its effectiveness in solving the Galerkin systems compared to the original multigrid solver, especially when the number of degrees of freedom associated with the spatial discretization is large.

Accelerating near-field 3D imaging approach for joint high-resolution imaging and phase error correction

The computational complexity and memory requirements of large-scale data seriously affect the application of compressed sensing (CS) in near-field three-dimensional (3-D) imaging system. In addition, as influenced by the measurement environment, the error in echo phase results in imaging defocusing. This paper proposes a CS near-field 3-D imaging approach based on nonuniform fast Fourier transform and phase error correction. It applies the fast Gaussian gridding nonuniform fast Fourier transform technique and Separable Surrogate Functionals with only matrix and vector multiplied to accelerate imaging speed and reduce memory requirements; it adopts the phase error correction technique to realize highly-focused imaging; in addition, a sparse observation approach based on Logistic sequence is proposed in this paper for easy availability of engineering realization for CS imaging. As indicated by numerical analysis and actual measurement in anechoic chamber, the approach proposed in this paper, compared with traditional imaging approaches, has the following advantages: accurate high resolution 3-D image of target can be obtained by applying small amount of observation data (10%); the computational complexity falls from O(LN) to O(3N) and memory occupation quantity drops from O(LN) to O(N); it can effectively perform highly-focused imaging for echo signal with phase error; the measurement matrix designed has better non-coherence and easy availability for engineering realization.

Maximizing the expected net present value of a project with phase-type distributed activity durations: An efficient globally optimal solution procedure

We study projects with activities that have stochastic durations that are modeled using phase-type distributions. Intermediate cash flows are incurred during the execution of the project. Upon completion of all project activities a payoff is obtained. Because activity durations are stochastic, activity starting times cannot be defined at the start of the project. Instead, we have to rely on a policy to schedule activities during the execution of the project. The optimal policy schedules activities such that the expected net present value of the project is maximized. We determine the optimal policy using a new continuous-time Markov chain and a backward stochastic dynamic program. Although the new continuous-time Markov chain allows to drastically reduce memory requirements (when compared to existing methods), it also allows activities to be preempted; an assumption that is not always desirable. We prove, however, that it is globally optimal not to preempt activities if cash flows are incurred at the start of an activity. Moreover, this proof holds regardless of the duration distribution of the activities. A computational experiment shows that we significantly outperform current state-of-the-art procedures. On average, we improve computational efficiency by a factor of 600, and reduce memory requirements by a factor of 321.

Boundary element method solution for large scale cathodic protection problems

Cathodic protection techniques are widely used for avoiding corrosion sequences in offshore structures. The Boundary Element Method (BEM) is an ideal method for solving such problems because requires only the meshing of the boundary and not the whole domain of the electrolyte as the Finite Element Method does. This advantage becomes more pronounced in cathodic protection systems since electrochemical reactions occur mainly on the surface of the metallic structure. The present work aims to solve numerically a sacrificial cathodic protection problem for a large offshore platform. The solution of that large-scale problem is accomplished by means of “PITHIA Software” a BEM package enhanced by Hierarchical Matrices (HM) and Adaptive Cross Approximation (ACA) techniques that accelerate drastically the computations and reduce memory requirements. The nonlinear polarization curves for steel and aluminium in seawater are employed as boundary condition for the under protection metallic surfaces and aluminium anodes, respectively. The potential as well as the current density at all the surface of the platform are effectively evaluated and presented.

Economically optimal safety targets for interdependent flood defences in a graph-based approach with an efficient evaluation of expected annual damage estimates

Abstract. Flood defence systems can be seen as multiple interdependent flood defences. This paper advances an approach for finding an optimal configuration for flood defence systems based on an economic cost–benefit analysis with an arbitrary number of interdependent flood defences. The proposed approach is based on a graph algorithm and is, thanks to some beneficial properties of the application, able to represent large graphs with strongly reduced memory requirements. Furthermore, computational efficiency is achieved by delaying cost calculations until they are actually needed by the graph algorithm. This significantly reduces the required number of computationally expensive flood risk calculations. In this paper, we conduct a number of case studies to compare the optimal paths found by the proposed approach with the results of competing methods that generate identical results. The proposed approach is set up in a generic way and implements the shortest-path approach for optimising cost–benefit analyses of interdependent flood defences with computationally expensive flood risk calculations.