Enormous Memory Requirements Research Articles

Implementation and optimization of hybrid parallel strategy in HNET

For the self-developed three-dimensional whole-core High-fidelity NEutron Transport calculation program HNET, although the numerical acceleration algorithms can improve the computation performance of high-fidelity neutron transport in terms of algorithms and models, it still faces some critical issues such as long computing time and enormous memory requirement. The rapid development of high-performance clusters provides a foundation for the application of massively parallel computing. Most current MOC programs are based on a single-type variable to achieve efficient parallelism, and spatial domain decomposition methods are the most common parallel schemes. However, its parallelism is limited and cannot fully utilize the current state-of-the-art computer resources. To solve this problem, hybrid parallel strategies are implemented in HNET to further expand the parallel degree, improve the speed of computation, and reduce the memory requirement. A hybrid MPI/OpenMP method based on domain decomposition and characteristic rays is proposed for the method of characteristics (MOC). For domain decomposition, the simulation domain is divided into spatial subdomains, with each subdomain handled by different processes. On this basis, the characteristic ray parallelism is implemented taking advantage of the inherent parallelism of the characteristic rays. Meanwhile, the optimization of the hybrid parallel strategy further improves the computation speed by eliminating atomic operations and using private pointer arrays and other techniques. In addition, in the framework of generalized equivalence theory (GET) based two-level coarse mesh finite difference method (CMFD), there is also a certain time consumption in solving CMFD linear system. Hence, for CMFD, a hybrid MPI/OpenMP method based on domain decomposition and secondary domain decomposition can be used. Using the secondary domain decomposition method, each subdomain is divided into sub-subdomains, with each sub-subdomain handled by threads, which allows CMFD to utilize the resources of characteristic ray parallelism in MOC, and also increases the speed of CMFD computation. Numerical results show that for both steady-state and transient calculations, the hybrid MPI/OpenMP in HNET can further expand the parallelism and accelerate the computation. It can take full use of parallel resources and achieve large-scale parallelism.

Robust extension of light fields with probable 3D distribution based on iterative scene estimation from multi-focus images

In this study, we propose an iterative approach to perform pseudo-scene reconstruction based on multi-focus images for estimating the 4D light fields of the scene robustly. An analogous scene-independent linear filtering approach has been proposed to efficiently achieve free viewpoint image reconstruction and scene refocusing with virtual bokeh by solely extracting 3D frequency components. However, it only succeeded in synthesizing limited regions of 4D light fields from multi-focus images, which can lead to awful degradation of reconstructed free viewpoint images outside the range due to the lack of the corresponding frequency components. In this paper, we directly utilize a linear relationship of 3D convolution combining multi-focus images and 3D scene by 3D blurring filters to estimate the 3D distribution itself of the scene instead of deriving the scene-independent linear filters from the relationship. In other words, we perform the scene reconstruction as a simplified convex optimization problem by modeling multi-focus images in multi-layered blurring composed of the scene and a bokeh filter. Algebraic reconstruction techniques of computed tomography, which uses the classical steepest descent method combined with projection onto convex sets (POCS) to ensure the convexity of the problem constraints, are introduced into such 3D scene reconstruction. The new method can be highly anticipated to assist us in extensively estimating robust 4D light fields. Experimental results illustrate that this approach enables good quality 3D image reconstruction with a wider range of 4D light fields. We also discuss the reduction of high computational costs and enormous memory requirements imposed by iterative algebraic reconstruction techniques. The quality of the obtained 4D light fields is comprehensively evaluated using both synthetic and real multi-focus images of practicable image resolution and depth range, additionally to an analysis of the faster processing times when comparing our iterative approach to the performance of the prior linear filters.

Evaluations on Deep Neural Networks Training Using Posit Number System

The training of Deep Neural Networks (DNNs) brings enormous memory requirements and computational complexity, which makes it a challenge to train DNN models on resource-constrained devices. Training DNNs with reduced-precision data representation is crucial to mitigate this problem. In this article, we conduct a thorough investigation on training DNNs with low-bit posit numbers, a Type-III universal number (Unum). Through a comprehensive analysis of quantization with various data formats, it is demonstrated that the posit format shows great potential to be employed in the training of DNNs. Moreover, a DNN training framework using 8-bit posit is proposed with a novel tensor-wise scaling scheme. The experiments show the same performance as the state-of-the-art (SOTA) across multiple datasets (MNIST, CIFAR-10, ImageNet, and Penn Treebank) and model architectures (LeNet-5, AlexNet, ResNet, MobileNet-V2, and LSTM). We further design an energy-efficient hardware prototype for our framework. Compared to the standard floating-point counterpart, our design achieves a reduction of 68, 51, and 75 percent in terms of area, power, and memory capacity, respectively.

A coupled monodomain solver with optimal memory usage for the simulation of cardiac wave propagation

The monodomain model is a common description for explaining the electrical activity of the heart. The model is nonlinear and consists of an ODE system that describes electrochemical reactions in the cardiac cells and a parabolic PDE that express the diffusion of the electrical signal. FEM approaches for the numerical solution of the monodomain equations can be classified into those that simultaneously update the state variables with respect to time by solving a coupled system and those that perform the temporal update in a decoupled manner. While coupled strategies are known to yield more accurate results, they so far have not been applied to physiological cell models due to their enormous memory requirements and computational times. In this paper, we tackle these challenges and suggest a novel computational strategy, that exploits the sparsity of local matrices in the assembly of global FEM matrices and hence features optimal usage of memory. We demonstrate the practicability of our coupled approach by employing three popular physiological cell models (Luo-Rudy phase-I, Ten Tusscher 2006 and O’Hara-Rudy 2011) and compare it with a commonly used decoupled strategy. Qualitatively different results obtained for the simulation of cardiac reentry underline the potential relevance of our work for future studies of cardiac pathology.

Accelerating Cartesian MRI by domain-transform manifold learning in phase-encoding direction.

This study developed a domain-transform framework comprising domain-transform manifold learning with an initial analytic transform to accelerate Cartesian magnetic resonance imaging (DOTA-MRI). The proposed method directly transforms undersampled Cartesian k-space data into a reconstructed image. In Cartesian undersampling, the k-space is fully or zero sampled in the data-acquisition direction (i.e., the frequency-encoding direction or the x-direction); one-dimensional (1D) inverse Fourier transform (IFT) along the x-direction on the undersampled k-space does not induce any aliasing. To exploit this, the algorithm first applies an analytic x-direction 1D IFT to the undersampled Cartesian k-space input, and subsequently transforms it into a reconstructed image using deep neural networks. The initial analytic transform (i.e., 1D IFT) allows the fully connected layers of the neural network to learn 1D global transform only in the phase-encoding direction (i.e., the y-direction) instead of 2D transform. This drastically reduces the number of parameters to be learned from O(N2) to O(N) compared with the existing manifold learning algorithm (i.e., automated transform by manifold approximation) (AUTOMAP). This enables DOTA-MRI to be applied to high-resolution MR datasets, which has previously proved difficult to implement in AUTOMAP because of the enormous memory requirements involved. After the initial analytic transform, the manifold learning phase uses a symmetric network architecture comprising three types of layers: front-end convolutional layers, fully connected layers for the 1D global transform, and back-end convolutional layers. The front-end convolutional layers take 1D IFT of the undersampled k-space (i.e., undersampled data in the intermediate domain or in the ky-x domain) as input and performs data-domain restoration. The following fully connected layers learn the 1D global transform between the ky-x domain and the image domain (i.e., the y-x domain). Finally, the back-end convolutional layers reconstruct the final image by denoising in the image domain. DOTA-MRI exhibited superior performance over nine other existing algorithms, including state-of-the-art deep learning-based algorithms. The generality of the algorithm was demonstrated by experiments conducted under various sampling ratios, datasets, and noise levels.

Compact in‐memory representation of large graph databases for efficient mining of maximal frequent sub graphs

SummaryComplex networks have been used in many scientific disciplines like sociology, microbiology, and telecommunication to represent the interactions among them. Graphs are generally used for representing such complex networks. Mining significant frequent patterns from graph databases has been a challenging area of research. A number of sub graph mining algorithms have been proposed for finding frequent fragments in molecular databases. A very few algorithms have been proposed for mining frequent patterns from large communication networks. All these algorithms perform well on medium size networks and fail on very large graphs. The scalability of these algorithms has been an issue because of the enormous memory requirements and also due to the exponential number of frequent sub graphs possible. In this paper, we propose a compact way of representing graph databases and also use it in a maximal frequent sub graph mining algorithm. The algorithm is found to be efficient and scalable to very large graph databases.

Key data for cloud computing based on ensemble clustering approximate analysis

To realise multi-label classification of text and meanwhile reduce calculation complexity and keep classification precision, dimensionality-reduction clustering method for fuzzy association of text multi-label based on cluster classification has been proposed. In text classification, it usually involves enormous feature numbers, which may cause curse of dimensionality. In addition, classification region can not always keep convex characteristics. It can be non-convex region composed of several overlapping or intersecting sub-regions. Above mentioned automatic classification system may require enormous memory requirement or has poor classification performance. Hence, new multi-label text classification method is proposed to overcome these problems in combination with fuzzy association technology. Fuzzy association evaluation is adopted to transform high-dimension text to low-dimension fuzzy association vector, thus avoiding curse of dimensionality. Experiment results show that the proposed method can more effectively classify text multi-label problem.

Key data for cloud computing based on ensemble clustering approximate analysis

To realize multi-label classification of text and meanwhile reduce calculation complexity and keep classification precision, dimensionality-reduction clustering method for fuzzy association of text multi-label based on cluster classification has been proposed. In text classification, it usually involves enormous feature numbers, which may cause curse of dimensionality. In addition, classification region can not always keep convex characteristics. It can be non-convex region composed of several overlapping or intersecting sub-regions. Abovementioned automatic classification system may require enormous memory requirement or has poor classification performance. Hence, new multi-label text classification method is proposed to overcome these problems in combination with fuzzy association technology. Fuzzy association evaluation is adopted to transform high-dimension text to low-dimension fuzzy association vector, thus avoiding curse of dimensionality. Experiment results show that the proposed method can more effectively classify text multi-label problem.

A unified implicit scheme for kinetic model equations. Part I. Memory reduction technique

A memory reduction technique is proposed for solving stationary kinetic model equations. As implied by an integral solution of the stationary kinetic equation, a velocity distribution function can be reconstructed from given macroscopic variables. Based on this fact, we propose a technique to reconstruct distribution function at discrete level, and employ it to develop an implicit numerical method for kinetic equations. The new implicit method only stores the macroscopic quantities which appear in the collision term, and does not store the distribution functions. As a result, enormous memory requirement for solving kinetic equations is totally relieved. Several boundary conditions, such as, inlet, outlet and isothermal boundaries, are discussed. Some numerical tests demonstrate the validity and efficiency of the technique. The new implicit solver provides nearly identical solution as the explicit kinetic solver, while the memory requirement is on the same order as the Navier–Stokes solver.

Fast solution of transient scattering from electrically large complex objects based on Time Domain Integral Equation solvers

A fast Time Domain Integral Equation (TDIE) solver is presented for analysis of transient scattering from electrically large conducting complex objects. The numerical process of Marching-On-in-Time (MOT) method based TDIE encounters high computational cost and exorbitant memory requirements. A group-style accelerated method-Plane Wave Time Domain (PWTD) algorithm, which permits rapid evaluation of transient wave field generated by temporally bandlimited sources, is employed to reduce the computational cost of MOT-based TDIE solvers. An efficient compressed storage technique for sparse matrix is adopted to decrease the enormous memory requirements of MOT. The scheme of the Multi-Level PWTD (MLPWTD)-enhanced MOT with compressed storage for sparse matrix is presented for analysis of transient scattering from electrically large complex objects in this paper. The numerical simulation results demonstrate the validity and efficiency of the presented scheme.

A self-organizing HCMAC neural-network classifier

This paper presents a self-organizing hierarchical cerebellar model arithmetic computer (HCMAC) neural-network classifier, which contains a self-organizing input space module and an HCMAC neural network. The conventional CMAC can be viewed as a basis function network (BFN) with supervised learning, and performs well in terms of its fast learning speed and local generalization capability for approximating nonlinear functions. However, the conventional CMAC has an enormous memory requirement for resolving high-dimensional classification problems, and its performance heavily depends on the approach of input space quantization. To solve these problems, this paper presents a novel supervised HCMAC neural network capable of resolving high-dimensional classification problems well. Also, in order to reduce what is often trial-and-error parameter searching for constructing memory allocation automatically, proposed herein is a self-organizing input space module that uses Shannon's entropy measure and the golden-section search method to appropriately determine the input space quantization according to the various distributions of training data sets. Experimental results indicate that the self-organizing HCMAC indeed has a fast learning ability and low memory requirement. It is a better performing network than the conventional CMAC for resolving high-dimensional classification problems. Furthermore, the self-organizing HCMAC classifier has a better classification ability than other compared classifiers.