Acceleration Techniques Research Articles

Accelerated musculoskeletal magnetic resonance imaging with deep learning-based image reconstruction at 0.55 T-3 T

Magnetic resonance imaging (MRI) is acentral component of musculoskeletal imaging. However, long image acquisition times can pose practical barriers in clinical practice. MRI is the established modality of choice in the diagnostic workup of injuries and diseases of the musculoskeletal system due to its high spatial resolution, excellent signal-to-noise ratio (SNR), and unparalleled soft tissue contrast. Continuous advances in hardware and software technology over the last few decades have enabled four-fold acceleration of 2D turbo-spin-echo (TSE) without compromising image quality or diagnostic performance. The recent clinical introduction of deep learning (DL)-based image reconstruction algorithms helps to minimize further the interdependency between SNR, spatial resolution and image acquisition time and allows the use of higher acceleration factors. The combined use of advanced acceleration techniques and DL-based image reconstruction holds enormous potential to maximize efficiency, patient comfort, access, and value of musculoskeletal MRI while maintaining excellent diagnostic accuracy. Accelerated MRI with DL-based image reconstruction has rapidly found its way into clinical practice and proven to be of added value. Furthermore, recent investigations suggest that the potential of this technology does not yet appear to be fully harvested. Deep learning-reconstructed fast musculoskeletal MRI examinations can be reliably used for diagnostic work-up and follow-up of musculoskeletal pathologies in clinical practice.

Topology Optimization of Self-supporting Structures for Additive Manufacturing via Implicit B-spline Representations

Owing to the rapid development in additive manufacturing, the potential to fabricate intricate structures has become a reality, emphasizing the importance of designing structures conducive to additive manufacturing processes. A crucial consideration is the ability to design structures requiring no additional support during manufacturing. This paper employs implicit B-spline representations for self-supporting structure design by integrating a topology optimization model with self-supporting constraints derived analytically from the implicit representation. This analytical derivation for detecting overhang regions enables accurate and efficient calculation of constraints, outperforming other B-spline-based methods. Compared to the traditional voxel-based methods, the implicit B-spline representation significantly expedites the optimization process by reducing the number of design variables. Additionally, several acceleration techniques are implemented to enhance the efficiency of our method, allowing simulations of 3D models with millions of finite elements to be completed within one and half an hour, excelling other B-spline-based methods and voxel-based methods. Various numerical experiments validate its excellent performance, confirming the effectiveness and efficiency of the proposed algorithm.

The generator distribution problem for base stations during emergency power outage: A branch-and-price-and-cut approach

Motivated by the need for uninterrupted service provision in the telecommunications industry, this paper presents a novel problem concerning the transportation of diesel generators during an unplanned power outage. Given a set of base stations, each equipped with a capacitated back-up battery pack, the problem consists in finding an optimal delivery and pick-up schedule that minimises corresponding costs. The problem is significantly complicated by the fact that the delivery quantity is a decision variable, and the pick-up times depend on the delivery time, delivery quantity and constraints related to battery-charging requirements. To address the issue, we develop problem-specific pick-up time functions that can be effectively incorporated into the proposed algorithm. The problem is formulated as an arc-based model and a set-packing model. To solve it to optimality, we design a tailored branch-and-price-and-cut algorithm, including novel customised dominance criteria in the labelling algorithm and adopting heuristics that exploit the problem features and effectively improve the algorithmic performance. The efficacy of the proposed algorithm is validated based on the extensive computational study and compared with a commercial solver. The proposed acceleration techniques are found to be potent with regard to computational times, and useful managerial insights are provided as part of sensitivity analyses.

Accelerated Stochastic Variance Reduction Gradient Algorithms for Robust Subspace Clustering.

Robust face clustering enjoys a wide range of applications for gate passes, surveillance systems and security analysis in embedded sensors. Nevertheless, existing algorithms have limitations in finding accurate clusters when data contain noise (e.g., occluded face clustering and recognition). It is known that in subspace clustering, the ℓ1- and ℓ2-norm regularizers can improve subspace preservation and connectivity, respectively, and the elastic net regularizer (i.e., the mixture of the ℓ1- and ℓ2-norms) provides a balance between the two properties. However, existing deterministic methods have high per iteration computational complexities, making them inapplicable to large-scale problems. To address this issue, this paper proposes the first accelerated stochastic variance reduction gradient (RASVRG) algorithm for robust subspace clustering. We also introduce a new momentum acceleration technique for the RASVRG algorithm. As a result of the involvement of this momentum, the RASVRG algorithm achieves both the best oracle complexity and the fastest convergence rate, and it reaches higher efficiency in practice for both strongly convex and not strongly convex models. Various experimental results show that the RASVRG algorithm outperformed existing state-of-the-art methods with elastic net and ℓ1-norm regularizers in terms of accuracy in most cases. As demonstrated on real-world face datasets with different manually added levels of pixel corruption and occlusion situations, the RASVRG algorithm achieved much better performance in terms of accuracy and robustness.

Domain decomposition methods and acceleration techniques for the phase field fracture staggered solver

AbstractThe phase field modeling of fracture is able to simulate the nucleation and the propagation of complex crack patterns. However, the relatively small internal lengths that are required usually lead to very fine meshes and high computational costs, especially for three‐dimensional applications. In the present work, additional cost also comes from the implicit dynamics regularization of unstable crack propagations which potentially leads to a large variation of time steps when switching from quasi‐static to dynamic regimes. To reduce the time to solution in this context, this study proposes a domain decomposition framework and acceleration techniques for the phase field fracture staggered solver. The displacement subproblem and the phase field one are solved with parallel domain decomposition solvers. Dual domain decomposition methods provide low cost preconditioner well adapted to the phase field subproblem. For displacement subproblems undergoing unstable crack propagations, primal domain decomposition methods are preferred to be less sensitive to the treatment of floating substructures. Preconditioners performances are assessed and scalability studies over academic test cases, up to 324 subdomains, are presented. Finally, the robustness of the approach is illustrated on two semi‐industrial simulations.

Multiobjective Evolutionary Generative Adversarial Network Compression for Image Translation

Generative Adversarial Networks (GANs) have achieved remarkable success in image translation tasks. However, its prohibitive computational overhead has been a major hurdle for deployment on resource constrained platforms. Due to the training instability and the complicated network architecture, existing model acceleration techniques cannot appropriately handle the GAN compression problem. To cope with these difficulties, we propose a multi-objective evolutionary algorithm to compress GAN models in the context of image translation tasks, which is termed as MEGC. Particularly, the conflict between the computational cost of the GAN model and the quality of generated image is explicitly modeled as a two-objective optimization problem, and the evolved Pareto set is utilized to guide the sampling process during supernet training, which can in turn divert the focus of the supernet training to well-performing compact subnets. Besides, an evaluation-free strategy is introduced to facilitate exploration in the search space while incurring no extra computational cost. Based on the above design, the proposed MEGC eliminates the requirement of subnet searching in the post-processing procedure. Experiments on image translation tasks under paired and unparied settings demonstrate the effectiveness of the proposed MEGC on reducing the computational cost of GANs while improving the quality of generated images compared to those of the full models.

Shuffle-QUDIO: accelerate distributed VQE with trainability enhancement and measurement reduction

The variational quantum eigensolver (VQE) is a leading strategy that exploits noisy intermediate-scale quantum (NISQ) machines to tackle chemical problems. It is expected to demonstrate quantum advantage when handling a large number of qubits, where the density matrix cannot be processed efficiently on classical computers. To gain such computational advantages on large-scale problems, a feasible solution is the QUantum DIstributed Optimization (QUDIO) scheme, which partitions the original problem into K\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{K}$$\\end{document} subproblems and allocates them to K\\documentclass[12pt]{minimal} \\usepackage{amsmath} \\usepackage{wasysym} \\usepackage{amsfonts} \\usepackage{amssymb} \\usepackage{amsbsy} \\usepackage{mathrsfs} \\usepackage{upgreek} \\setlength{\\oddsidemargin}{-69pt} \\begin{document}$$\\varvec{K}$$\\end{document} quantum machines followed by the parallel optimization. Despite the provable acceleration ratio, the efficiency of QUDIO may heavily degrade by the synchronization operation. To conquer this issue, here we propose Shuffle-QUDIO to involve shuffle operations into local Hamiltonians during the quantum distributed optimization. Compared with QUDIO, Shuffle-QUDIO significantly reduces the communication frequency among quantum processors and simultaneously achieves better trainability. Particularly, we prove that Shuffle-QUDIO enables a faster convergence rate over QUDIO. Extensive numerical experiments are conducted to verify that Shuffle-QUDIO allows both a wall-clock time speedup and low approximation error in the tasks of estimating the ground state energy of a molecule. We empirically demonstrate that our proposal can be seamlessly integrated with other acceleration techniques, such as operator grouping, to further improve the efficacy of VQE.

Accelerated evaluation of blocking flowshop scheduling with total flow time criteria using a generalized critical machine-based approach

Despite the considerable advances in the research of the blocking flowshop scheduling problem (BFSP), several unresolved challenges persist. Algorithmic complexity presents hurdles. Although the insertion-based method is considered to generate superior solutions, its high computational demand diminishes the efficiency of algorithms, especially within large-scale sequences. The existing accelerated evaluation methods cannot utilize the existing information to quickly calculate the total flow time or the total tardiness time of the changed sequence after the job insertion, but recalculates it from scratch. This does not significantly reduce computational effort and needs to be further improved.In this paper, we delve into the intrinsic features of these challenges, proposing a generalized accelerated critical machine-based evaluation tailored for the total flow time and tardiness criteria of the BFSP with and without sequence-dependent setup times. First, we propose three theorems, one corollary, and their proofs based on the critical machine. Second, we propose the accelerated evaluation procedure based on these theorems to calculate the objectives related to the total flow time. Third, we also extend the proposed accelerated evaluation method to the BFSP with sequence-dependent setup times, aiming to significantly reduce the time complexity. Finally, we conduct four experiments on five well-known benchmarks (a total of 3540 test instances). Through statistical analysis, it becomes evident that our computational efforts have significantly decreased in computing both the total flow time and the total tardiness time. This performance enhancement is superior to the effectiveness of existing acceleration techniques.

Electromagnetic scattering and imaging simulation of extremely large-scale sea-ship scene based on GPU parallel technology

Aiming to solve the bottleneck problem of electromagnetic scattering simulation in the scenes of extremely large-scale seas and ships, a high-frequency method by using graphics processing unit (GPU) parallel acceleration technique is proposed. For the implementation of different electromagnetic methods of physical optics (PO), shooting and bouncing ray (SBR) and physical theory of diffraction (PTD), a parallel computing scheme based on the central processing unit (CPU)-GPU parallel computing scheme is realized to balance computing tasks. Finally, a multi-GPU framework is further proposed to solve the computational difficulty caused by the massive number of ray tubes in the ray tracing process. By using the established simulation platform, signals of ships at different seas are simulated and their images are achieved as well. It is shown that the higher sea states degrade the averaged peak signal-to-noise ratio (PSNR) of radar image.

Recovery time reduction to decrease experimental duration (R2D2): A simple and universal method to accelerate NMR experiments

Acceleration techniques for one dimensional Nuclear Magnetic Resonance (1D NMR) are very useful, both for NMR enthusiasts and for chemists that use NMR for structural elucidation. To the latter, such techniques need to be straightforward. Recovery time Reduction to Decrease the experimental Duration (R2D2) relies on the incremental reduction of a pulse sequence's Recycle Time (TR). A pseudo-2D spectrum is acquired and after two Fourier transform, extraction and addition of the central rows, a 1D spectrum is obtained. Not only can it be applied to any pulse sequence that contains a TR, but it also requires only a list of recovery times and 2D processes to operate. With this method, we were able to easily reduce the experimental time by a factor of 2 and up to 4 using single-pulse, APT and DEPT 13C sequences. Moreover, R2D2 has the potential to be used on other low abundance nuclei (such as 15N or 2H) and numerous other pulse sequences.

Accelerated Benders decomposition and local branching for dynamic maximum covering location problems

The maximum covering location problem (MCLP) is a key problem in facility location, with many applications and variants. One such variant is the dynamic (or multi-period) MCLP, which considers the installation of facilities across multiple time periods. To the best of our knowledge, no exact solution method has been proposed to tackle large-scale instances of this problem. To that end, in this work, we expand upon the current state-of-the-art branch-and-Benders-cut solution method in the static case, by exploring several acceleration techniques. Additionally, we propose a specialised local branching scheme which exploits the separability of the problem by time period. This scheme uses a novel distance metric in its definition of subproblems and features a new method for efficient and exact solving of the subproblems. These methods are then compared through extensive computational experiments, highlighting the strengths of the proposed methodologies.

Multilayer Neural Network Hardware Designs for Faster Convolutional Processing

The demand for efficient hardware implementations of convolutional neural networks (CNNs) has surged with the proliferation of deep learning applications in various domains such as computer vision, natural language processing, and autonomous systems. Convolutional layers, being the fundamental building blocks of CNNs, are computationally intensive, requiring optimized hardware architectures for real-time inference tasks. This paper presents novel hardware designs targeting faster convolutional processing within multilayer neural networks. We propose a multi-faceted approach that leverages parallelism, pipelining, and hardware acceleration techniques to enhance the efficiency of convolution operations. Our design optimally exploits the inherent data-level and model-level parallelism present in CNNs to achieve high throughput while minimizing latency.

Resource-aware Montgomery modular multiplication optimization for digital signal processing

Homomorphic encryption is an important technology for protecting data privacy, and the performance of modular multiplication directly affects the efficiency of homomorphic encryption. Currently, there are numerous FPGA-based acceleration techniques targeting modular multiplication. However, many of these implementations require substantial hardware resources or suffer from resource imbalance. This leads to a lower throughput. Therefore, we present a novel FPGA-based implementation of Montgomery Modular Multiplication aimed at addressing these challenges. Our design employs a suitable radix bit width and word size based on the digital signal processing (DSP) bit width rather than the conventional binary powers of two. We aim to instantiate more modular multipliers using limited resources while minimizing latency. We also introduce a novel DSP cascade structure, called parallel grouping cascade DSP, which reduces the number of clock cycles of internal multipliers. To balance the ratio of lookup table (LUT) and DSP usage, we also use multipliers implemented in the LUT to replace some DSPs. Our results, implemented on Xilinx Virtex-7 field-programmable gate array (FPGA), demonstrate more than 27% improvement in throughput on 1024-bit modular multiplication and more than 70% improvement on 2048-bit compared to the best previous state-of-the-art references.

Finite element iterative algorithm based on Anderson acceleration technique for incompressible MHD equations

In this paper, an Anderson’s acceleration algorithm for the finite element Oseen iterative scheme of the steady incompressible magnetohydrodynamics (MHD) problem is designed and analyzed. The Anderson acceleration algorithm based on solving an optimization problem in each iteration receives great attention to accelerate linear and nonlinear iterations. We prove that the proposed algorithm has higher convergence rate than that of the Oseen iterative method for the steady incompressible MHD, and give the accelerating convergence theory under different acceleration depths m. Numerical experiments including the singular solution and benchmark problems are carried out to verify the correctness and effectiveness of the algorithm. The tests show the marked improvement in the high physical parameters (including the hydrodynamic and magnetic Reynolds numbers, and the coupling number) when the Oseen iteration fails to converge.

On two-term hypergeometric recursions with free lower parameters

Let F ( n , k ) be a hypergeometric function that may be expressed so that n appears within initial arguments of inverted Pochhammer symbols, as in factors of the form 1 ( n ) k . Only in exceptional cases is F ( n , k ) such that Zeilberger's algorithm produces a two-term recursion for ∑ k = 0 ∞ F ( n , k ) obtained via the telescoping of the right-hand side of a difference equation of the form p 1 ( n ) F ( n + r , k ) + p 2 ( n ) F ( n , k ) = G ( n , k + 1 ) − G ( n , k ) for fixed r ∈ N and polynomials p 1 and p 2 . Building on the work of Wilf, we apply a series acceleration technique based on two-term hypergeometric recursions derived via Zeilberger's algorithm. Fast converging series previously given by Ramanujan, Guillera, Chu and Zhang, Chu, Lupaş, and Amdeberhan are special cases of hypergeometric transforms introduced in our article.

Viscoelastic flow with slip in a hyperbolic channel

We study theoretically the steady viscoelastic flow in confined and symmetric hyperbolic channels considering slip along the walls. Under the lubrication approximation and a variety of constitutive models, a high-order perturbation solution with respect to the Deborah number is calculated. The solution for all the field variables (velocity, pressure, and extra-stress) is found analytically up to eighth order and is used along with proper acceleration techniques to achieve convergence up to order one Deborah number. We reveal that even in the presence of slip, the pressure drop decreases monotonically with increasing the fluid elasticity. We evaluate the influence of slip in terms arising from two different decompositions of the pressure drop obtained with the aid of the total force balance and the mechanical energy balance of the flow system. In contrast to the nonslip Newtonian flow, our analysis also showed that the fluid slip along the walls introduces variations in the strain rate at the midplane with the distance from the inlet. However, these are small, and an effective strain rate can be well-represented using a previously developed formula [Housiadas, K. D., and A. N. Beris, Phys. Fluids 36(2), 021702 (2024)]. We also show that when the solution for the midplane velocity is used in the general formula for the Trouton ratio, instead of the Newtonian lubrication solution, there are no appreciable changes, thus confirming the validity and accuracy of our previously reported results [Housiadas, K. D., and A. N. Beris, J. Rheol. 68(3), 327–339 (2024)].

A comparative study on the acceleration techniques for solving finite difference discretization poisson’s equation in the PIC/MCC Method

A wide variety of plasma phenomena have been investigated during the past decades using the particle-in-cell/Monte Carlo collisions (PIC/MCC) method. As an important component of the PIC/MCC method, solving Poisson’s equation is crucial for the accuracy and efficiency of calculations. Different acceleration techniques for solving finite difference discretization Poisson’s equation are investigated and compared, including direct method, iterative method, multigrid (MG) method, parallel computing and inherited initial value. The charge density distribution with a known analytical solution is used to validate the algorithm and code. The optimal relaxation factor for the successive over-relaxation (SOR) method in 2D Poisson’s equation with unequal grid node numbers in different dimensions is derived, which is only related to the dimension with the largest grid number. Although there will be a ‘more optimal’ relaxation factor deviated from in some simulation cases, selecting the optimal relaxation factor derived always leads to a not slow solving speed. However, when SOR is used in MG for smoothing, the optimal relaxation factor will shift to 0.5–1.2 from the theoretical optimal value derived with the increase of MG levels. By comparing the convergence order under different relaxation factors and MG levels, the suitable MG level is proposed as log2[min(N x , N y )]−2. Combining the optimal SOR relaxation factor, MG, parallel computing and inherited initial values, the computational cost may decrease by 5 orders of magnitude than that by the simple Gaussian elimination (GE). Based on the optimal acceleration techniques mentioned above, a benchmark simulation case electron cyclotron drift instability (ECDI) in magnetized plasmas was run to further validate the developed PIC/MCC code. The distributions of electric field in the x-direction, electron density and electron temperature are all consistent with the literatures. This paper provides a reference for the acceleration strategy selection for solving Poisson’s equation quickly in plasma simulations.

Accelerated Double-Sketching Subspace Newton

This paper proposes a second-order stochastic algorithm called Accelerated Double-Sketching Subspace Newton (ADSSN) to solve large-scale optimization problems with high dimensional feature spaces and substantial sample sizes. The proposed ADSSN has two computational superiority. First, ADSSN achieves a fast local convergence rate by exploiting Nesterov’s acceleration technique. Second, by taking full advantage of the double sketching strategy, ADSSN provides a lower computational cost for each iteration than competitive approaches. Moreover, these advantages hold for actually all sketching techniques, which enables practitioners to design custom sketching methods for specific applications. Finally, numerical experiments are carried out to demonstrate the efficiency of ADSSN compared with accelerated gradient descent and two single sketching counterparts.

Deep unrolling networks with recurrent momentum acceleration for nonlinear inverse problems

Combining the strengths of model-based iterative algorithms and data-driven deep learning solutions, deep unrolling networks (DuNets) have become a popular tool to solve inverse imaging problems. Although DuNets have been successfully applied to many linear inverse problems, their performance tends to be impaired by nonlinear problems. Inspired by momentum acceleration techniques that are often used in optimization algorithms, we propose a recurrent momentum acceleration (RMA) framework that uses a long short-term memory recurrent neural network (LSTM-RNN) to simulate the momentum acceleration process. The RMA module leverages the ability of the LSTM-RNN to learn and retain knowledge from the previous gradients. We apply RMA to two popular DuNets—the learned proximal gradient descent (LPGD) and the learned primal-dual (LPD) methods, resulting in LPGD-RMA and LPD-RMA, respectively. We provide experimental results on two nonlinear inverse problems: a nonlinear deconvolution problem, and an electrical impedance tomography problem with limited boundary measurements. In the first experiment we have observed that the improvement due to RMA largely increases with respect to the nonlinearity of the problem. The results of the second example further demonstrate that the RMA schemes can significantly improve the performance of DuNets in strongly ill-posed problems.

Toward Quantized Model Parallelism for Graph-Augmented MLPs Based on Gradient-Free ADMM Framework.

While graph neural networks (GNNs) are popular in the deep learning community, they suffer from several challenges including over-smoothing, over-squashing, and gradient vanishing. Recently, a series of models have attempted to relieve these issues by first augmenting the node features and then imposing node-wise functions based on multilayer perceptron (MLP), which are widely referred to as graph-augmented MLP (GA-MLP) models. However, while GA-MLP models enjoy deeper architectures for better accuracy, their efficiency largely deteriorates. Moreover, popular acceleration techniques such as stochastic-version or data-parallelism cannot be effectively applied due to the dependency among samples (i.e., nodes) in graphs. To address these issues, in this article, instead of data parallelism, we propose a parallel graph deep learning Alternating Direction Method of Multipliers (pdADMM-G) framework to achieve model parallelism: parameters in each layer of GA-MLP models can be updated in parallel. The extended pdADMM-G-Q algorithm reduces communication costs by introducing the quantization technique. Theoretical convergence to a (quantized) stationary point of the pdADMM-G algorithm and the pdADMM-G-Q algorithm is provided with a sublinear convergence rate o(1/k) , where k is the number of iterations. Extensive experiments demonstrate the convergence of two proposed algorithms. Moreover, they lead to a more massive speedup and better performance than all state-of-the-art comparison methods on nine benchmark datasets. Last but not least, the proposed pdADMM-G-Q algorithm reduces communication overheads by up to 45% without loss of performance. Our code is available at https://github.com/xianggebenben/pdADMM-G.