Multiple Architectures Research Articles

Zero time waste in pre-trained early exit neural networks

The problem of reducing processing time of large deep learning models is a fundamental challenge in many real-world applications. Early exit methods strive towards this goal by attaching additional Internal Classifiers (ICs) to intermediate layers of a neural network. ICs can quickly return predictions for easy examples and, as a result, reduce the average inference time of the whole model. However, if a particular IC does not decide to return an answer early, its predictions are discarded, with its computations effectively being wasted. To solve this issue, we introduce Zero Time Waste (ZTW), a novel approach in which each IC reuses predictions returned by its predecessors by (1) adding direct connections between ICs and (2) combining previous outputs in an ensemble-like manner. We conduct extensive experiments across various multiple modes, datasets, and architectures to demonstrate that ZTW achieves a significantly better accuracy vs. inference time trade-off than other early exit methods. On the ImageNet dataset, it obtains superior results over the best baseline method in 11 out of 16 cases, reaching up to 5 percentage points of improvement on low computational budgets.

Area, power efficient Vedic multiplier architecture using novel 4:2 compressor

Area, power efficient Vedic multiplier architecture using novel 4:2 compressor

Comparison of RNN Architectures and Non-RNN Architectures in Sentiment Analysis

This study compares the sentiment analysis performance of multiple Recurrent Neural Network architectures and One-Dimensional Convolutional Neural Networks. THE METHODS EVALUATED ARE simple Recurrent Neural Network, Long Short-Term Memory, Gated Recurrent Unit, Bidirectional Recurrent Neural Network, and 1D ConvNets. A dataset comprising text reviews with positive or negative sentiment labels was evaluated. All evaluated models demonstrated an extremely high accuracy, ranging from 99.81% to 99.99%. Apart from that, the loss generated by these models is also low, ranging from 0.0043 to 0.0021. However, there are minor performance differences between the evaluated architectures. The Long Short-Term Memory and Gated Recurrent Unit models mainly perform marginally better than the Simple Recurrent Neural Network, albeit with slightly lower accuracy and loss. In the meantime, the Bidirectional Recurrent Neural Network model demonstrates competitive performance, as it can effectively manage text context from both directions. Additionally, One-Dimensional Convolutional Neural Networks provide satisfactory results, indicating that convolution-based approaches are also effective in sentiment analysis. The findings of this study provide practitioners with essential insights for selecting an appropriate architecture for sentiment analysis tasks. While all models yield excellent performance, the choice of architecture can impact computational efficiency and training time. Therefore, a comprehensive comprehension of the respective characteristics of Recurrent Neural Network architectures and One-Dimensional Convolutional Neural Networks is essential for making more informed decisions when constructing sentiment analysis models.

A novel approach and hybrid parallel algorithms for solving the fixed charge transportation problem

This article is dedicated to the efficient resolution of the fixed charge transport problem (FCTP) with the goal of identifying optimal solutions within reduced timeframes. FCTP is a combinatorial and NP-complete problem known for its exponential time complexity relative to problem size. Metaheuristic methods, including genetic algorithms, represent effective techniques for obtaining high-quality FCTP solutions. Consequently, the integration of parallel algorithms emerges as a strategy for expediting problem-solving. The proposed approach, referred to as the parallel genetic algorithm (PGA), entails the application of a genetic algorithm across multiple parallel architectures to tackle the FCTP problem. The primary aim is to explore fresh solutions for the fixed charge transportation problem using genetic algorithms while concurrently optimizing the time required to achieve these solutions through parallelism. The FCTP problem is fundamentally a linear programming challenge, revolving around the determination of optimal shipment quantities from numerous source locations to multiple destinations with the overarching objective of minimizing overall transportation costs. This necessitates consideration of constraints tied to product availability at the sources and demand dynamics at the destinations. In this study, a pioneering approach to addressing the Fixed Charge Transportation Problem (FCTP) using parallel genetic algorithms (PGA) is unveiled. The research introduces two distinct parallel algorithms: The Master-Slave Approach (MS-GA) and the Coarse-Grained Approach (CG-GA). Additionally, investigation into the hybridization of these approaches has led to the development of the NMS-CG-GA approach. The numerical results reveal that our parallelism-based approaches significantly improve the performance of genetic algorithms. Specifically, the Master-Slave (MS-GA) approach demonstrates its advantages in solving smaller instances of the FCTP problem, while the Coarse-Grained (CG-GA) approach exhibits greater effectiveness for larger problem instances. The conclusion reached is that the novel hybrid parallel genetic algorithm approach (NMS-CG-GA) outperforms its predecessors, yielding outstanding results, particularly across diverse FCTP problem instances.

On the Use of Low-power Devices, Approximate Adders and Near-threshold Operation for Energy-efficient Multipliers

With the rising importance of power consumption in battery-powered devices, approximate computing techniques have emerged as a promising approach to strike a balance between exact computation and power savings, leading to improved delays. This paper investigates the combination of near-threshold operation and approximate adders to design power-efficient multipliers. We analyzed four multiplier architectures using 16 nm low-power and high-performance models. At the transistor level, three strategies for approximate full adders are explored, focusing on both partial product reduction and the final addition stage of the multipliers. Eleven test cases are thoroughly evaluated to identify the most suitable approximate circuit, considering the trade-offs among power, performance, and accuracy. The obtained results demonstrate a substantial reduction in power consumption at near-threshold operation. The replacement of exact full adders with the approximate copy strategy in the least significant bits of the multipliers leads to a reduction of up to 34.4% in power consumption and 19.2% in delay. The design-space exploration carried out in this study provides valuable insights for designers to choose the best approximate multiplier based on specific design requirements.

Startup Competitions Decoded: Unpacking the Phenomenon's Dimensions

Drawing on a typological theorizing style (Cornelissen, 2017), this paper investigates and deconstructs the fuzzy nature of startup competitions (Cornelissen, Höllerer and Seidl, 2021). Despite the pivotal role that startup competitions play in promoting innovation and shaping policies related to business creation and welfare there exists a scarcity of comprehensive research that thoroughly investigates the organizational architecture and overall effects of these contests on entrepreneurial ecosystems’ actors. Furthermore, the presence of conceptual fragmentation necessitates a process of synthesis to prevent the dispersion of scientific efforts and to enhance understanding of the essence and constituents of the phenomenon (Ferreira, Fernandes and Ratten, 2016; Howard and Crayne, 2019; Law, Wong and Mobley, 1998). The methodology utilized in this study is a validated approach that combines a systematic literature review with subsequent inductive analysis of the selected articles (Thomas and Tee, 2022). This article makes two distinct contributions to the existing body of literature. First, this study contributes to the startup competitions and entrepreneurship literature by providing a typological theoretical framework that examines the conceptual dimensions of this phenomenon. Additionally, this research seeks to identify the key actors involved in these competitions and emphasize the potential of utilizing such events as intervention tools at micro and macro-organizational levels. Furthermore, this study makes a valuable contribution to the existing body of literature that examines the intersection of entrepreneurship and organizational studies. This study presents a theoretical framework that focuses on the unique characteristics of organizational actors involved. It illustrates the potential design of multiple SUCs architectures based on the objectives and deliberate impacts that these actors aim to achieve. From the perspective of managerial implications, this work provides fuel for thought and a practical instrument for all organizations that intend to conduct a startup competition on purpose. This paper functions as an in-depth guide for all relevant stakeholders, including organizers, participants, accelerators, incubators, and other involved actors. This phenomenon illustrates how each actor is prompted to take action upon acquiring a specific scope.

Implementation of Efficient Vedic Multiplier and Its Performance Evaluation

The ancient Vedic mathematics is well known for quicker handy multiplications but its recognition as an integrated circuit core against existing hardware multipliers is not established. As optimized hardware implementation of binary multiplier is one of the prominent unsolved problems in computer architecture, this paper proposes efficient Urdhava Tiryakbhyam Vedic multiplier architecture and compares it with the set of hierarchical multiplication algorithms which generate multiplication result in a single clock cycle. Two innovative algorithms are proposed here, one with a compact structure and another for faster execution. Also, its optimized transistor level layout is designed and implemented. To maintain homogeneity for comparison, all the algorithms are programmed on a common HDL language platform and analyzed with the same tool and technology. Final results indicate that the proposed architecture delivers 15.5% less power delay product (PDP) compared to closest competitor algorithm.

High-Speed VLSI Architectures for Modular Polynomial Multiplication via Fast Filtering and Applications to Lattice-Based Cryptography

This paper presents a low-latency hardware accelerator for modular polynomial multiplication for lattice-based post-quantum cryptography and homomorphic encryption applications. The proposed novel modular polynomial multiplier exploits the fast finite impulse response (FIR) filter architecture to reduce the computational complexity of the schoolbook modular polynomial multiplication. We also extend this structure to fast $M$-parallel architectures while achieving low-latency, high-speed, and full hardware utilization. We comprehensively evaluate the performance of the proposed architectures under various polynomial settings as well as in the Saber scheme for post-quantum cryptography as a case study. The experimental results show that our proposed modular polynomial multiplier reduces the computation time and area-time product, respectively, compared to the state-of-the-art designs.

Topgun: An ECC Accelerator for Private Set Intersection

Elliptic Curve Cryptography (ECC), one of the most widely used asymmetric cryptographic algorithms, has been deployed in Transport Layer Security (TLS) protocol, blockchain, secure multiparty computation, and so on. As one of the most secure ECC curves, Curve25519 is employed by some secure protocols, such as TLS 1.3 and Diffie-Hellman Private Set Intersection (DH-PSI) protocol. High-performance implementation of ECC is required, especially for the DH-PSI protocol used in privacy-preserving platform. Point multiplication, the chief cryptographic primitive in ECC, is computationally expensive. To improve the performance of DH-PSI protocol, we propose Topgun, a novel and high-performance hardware architecture for point multiplication over Curve25519. The proposed architecture features a pipelined Finite-field Arithmetic Unit and a simple and highly efficient instruction set architecture. Compared to the best existing work on Xilinx Zynq 7000 series FPGA, our implementation with one Processing Element can achieve 3.14× speedup on the same device. To the best of our knowledge, our implementation appears to be the fastest among the state-of-the-art works. We also have implemented our architecture consisting of 4 Compute Groups, each with 16 PEs, on an Intel Agilex AGF027 FPGA. The measured performance of 4.48 Mops/s is achieved at the cost of 86 Watts power, which is the record-setting performance for point multiplication over Curve25519 on FPGAs.

Force Profile as Surgeon-Specific Signature.

To investigate the notion that a surgeon's force profile can be the signature of their identity and performance. Surgeon performance in the operating room is an understudied topic. The advent of deep learning methods paired with a sensorized surgical device presents an opportunity to incorporate quantitative insight into surgical performance and processes. Using a device called the SmartForceps System and through automated analytics, we have previously reported surgeon force profile, surgical skill, and task classification. However, an investigation of whether an individual surgeon can be identified by surgical technique has yet to be studied. In this study, we investigate multiple neural network architectures to identify the surgeon associated with their time-series tool-tissue forces using bipolar forceps data. The surgeon associated with each 10-second window of force data was labeled, and the data were randomly split into 80% for model training and validation (10% validation) and 20% for testing. Data imbalance was mitigated through subsampling from more populated classes with a random size adjustment based on 0.1% of sample counts in the respective class. An exploratory analysis of force segments was performed to investigate underlying patterns differentiating individual surgical techniques. In a dataset of 2819 ten-second time segments from 89 neurosurgical cases, the best-performing model achieved a micro-average area under the curve of 0.97, a testing F1-score of 0.82, a sensitivity of 82%, and a precision of 82%. This model was a time-series ResNet model to extract features from the time-series data followed by a linearized output into the XGBoost algorithm. Furthermore, we found that convolutional neural networks outperformed long short-term memory networks in performance and speed. Using a weighted average approach, an ensemble model was able to identify an expert surgeon with 83.8% accuracy using a validation dataset. Our results demonstrate that each surgeon has a unique force profile amenable to identification using deep learning methods. We anticipate our models will enable a quantitative framework to provide bespoke feedback to surgeons and to track their skill progression longitudinally. Furthermore, the ability to recognize individual surgeons introduces the mechanism of correlating outcome to surgeon performance.

Machine learning enhanced droplet microfluidics

Machine learning has recently been introduced in the context of droplet microfluidics to simplify the process of droplet formation, which is usually controlled by a variety of parameters. However, the studies introduced so far have mainly focused on droplet size control using water and mineral oil in microfluidic devices fabricated using soft lithography or rapid prototyping. This approach negated the applicability of machine learning results to other types of fluids more relevant to biomedical applications, while also preventing users that do not have access to microfluidic fabrication facilities to take advantage of previous findings. There are a number of different algorithms that could be used as part of a data driven approach, and no clear comparison has been previously offered among multiple machine learning architectures with respect to the predictions of flow rate values and generation rate. We here employed machine learning to predict the experimental parameters required for droplet generation in three commercialized microfluidic flow-focusing devices using phosphate buffer saline and biocompatible fluorinated oil as dispersed and continuous liquid phases, respectively. We compared three different machine learning architectures and established the one leading to more accurate predictions. We also compared the predictions with a new set of experiments performed at a different day to account for experimental variability. Finally, we provided a proof of concept related to algae encapsulation and designed a simple app that can be used to generate accurate predictions for a given droplet size and generation rate across the three commercial devices.

Video Relay Service Recording System Using Packet-Capture Multiple Buckets Architecture

This paper presents a research and development work for the automatic service recording system using network-based monitoring approach. The project is under the cooperation of National Science and Technology Development Agency (NSTDA) and Thai Telecommunication Relay Service (TTRS). The goal is to archive and record video relay services offered to the deaf and hard of hearing community. Using the network-based monitoring approach, we can develop a video recording system that is fully automated and intelligent to analyze different kinds of video relay services. In brief, the system functions to capture network traffic and analyze network packets, signaling, and related protocols such as TCP, UDP, SIP, SDP, RTP, as well as H.264, G.711 encoding. Then, it records the relevant media streams, creates a video file containing the whole video relay service conversation, and send the file to archive in the cloud storage. The benefits include verification in case of appeal, service evaluation, and service improvement. A technical challenge in this system development is how to process real-time interactive multi flows because it must detect and process a large number of call signaling and media streams that occur as interactive real time. The optimization is vital so that the system can swiftly do the processing in time, without causing the buffer overflow problem that can degrade the recording quality. In this regard, we developed a novel methodology for processing PCAP (Packet Capture) called Multiple Buckets Architecture. Our test results showed that it can significantly reduce the probability of buffer overflow, making the processing more efficient when compared to the traditional processing of single bucket.

HAPNEST: efficient, large-scale generation and evaluation of synthetic datasets for genotypes and phenotypes.

Existing methods for simulating synthetic genotype and phenotype datasets have limited scalability, constraining their usability for large-scale analyses. Moreover, a systematic approach for evaluating synthetic data quality and a benchmark synthetic dataset for developing and evaluating methods for polygenic risk scores are lacking. We present HAPNEST, a novel approach for efficiently generating diverse individual-level genotypic and phenotypic data. In comparison to alternative methods, HAPNEST shows faster computational speed and a lower degree of relatedness with reference panels, while generating datasets that preserve key statistical properties of real data. These desirable synthetic data properties enabled us to generate 6.8 million common variants and nine phenotypes with varying degrees of heritability and polygenicity across 1 million individuals. We demonstrate how HAPNEST can facilitate biobank-scale analyses through the comparison of seven methods to generate polygenic risk scoring across multiple ancestry groups and different genetic architectures. A synthetic dataset of 1 008 000 individuals and nine traits for 6.8 million common variants is available at https://www.ebi.ac.uk/biostudies/studies/S-BSST936. The HAPNEST software for generating synthetic datasets is available as Docker/Singularity containers and open source Julia and C code at https://github.com/intervene-EU-H2020/synthetic_data.

Yes-associated protein regulates cortical actin architecture and dynamics through intracellular translocation of Rho GTPase-activating protein 18.

Yes-associated protein (YAP) is a transcriptional co-activator that controls the transcription of target genes and modulates the structures of various cytoskeletal architecture as mechanical responses. Although it has been known that YAP regulates actin-regulatory proteins, the detailed molecular mechanism of how they control and coordinate intracellular actin architecture remains elusive. Herein, we aimed to examine the structure and dynamics of intracellular actin architecture from molecular to cellular scales in normal and YAP-knockout (YAP-KO) cells utilizing high-speed atomic force microscopy (HS-AFM) for live-cell imaging and other microscope-based mechanical manipulation and measurement techniques. YAP-KO Madin-Darby canine kidney cells had a higher density and turnover of actin filaments in the cell cortex and a higher elastic modulus. Laser aberration assay demonstrated that YAP-KO cells were more resistant to damage than normal cells. We also found that Rho GTPase-activating protein 18 (ARHGAP18), a downstream factor of YAP, translocated from the cortex to the edge of sparsely cultured YAP-KO cells. It resulted in high RhoA activity and promotion of actin polymerization in the cell cortex and their reductions at the edge. HS-AFM imaging of live cell edge and a cell-migration assay demonstrated lower membrane dynamics and motility of YAP-KO cells than those of normal cells, suggesting lower actin dynamics at the edge. Together, these results demonstrate that a YAP-dependent pathway changes the intracellular distribution of RhoGAP and modulates actin dynamics in different parts of the cell, providing a mechanistic insight into how a mechano-sensitive transcription cofactor regulates multiple intracellular actin architecture and coordinates mechano-responses.

Throughput/Area-Efficient Accelerator of Elliptic Curve Point Multiplication over GF(2233) on FPGA

This paper presents a throughput/area-efficient hardware accelerator architecture for elliptic curve point multiplication (ECPM) computation over GF(2233). The throughput of the proposed accelerator design is optimized by reducing the total clock cycles using a bit-parallel Karatsuba modular multiplier. We employ two techniques to minimize the hardware resources: (i) a consolidated arithmetic unit where we combine a single modular adder, multiplier, and square block instead of having multiple modular operators, and (ii) an Itoh–Tsujii inversion algorithm by leveraging the existing hardware resources of the multiplier and square units for multiplicative inverse computation. An efficient finite-state-machine (FSM) controller is implemented to facilitate control functionalities. To evaluate and compare the results of the proposed accelerator architecture against state-of-the-art solutions, a figure-of-merit (FoM) metric in terms of throughput/area is defined. The implementation results after post-place-and-route simulation are reported for reconfigurable field-programmable gate array (FPGA) devices. Particular to Virtex-7 FPGA, the accelerator utilizes 3584 slices, needs 7208 clock cycles, operates on a maximum frequency of 350 MHz, computes one ECPM operation in 20.59 μs, and the calculated value of FoM is 13.54. Consequently, the results and comparisons reveal that our accelerator suits applications that demand throughput and area-optimized ECPM implementations.

W-AMA: Weight-aware Approximate Multiplication Architecture for neural processing

This paper presents the Weight-aware Approximate Multiplication Architecture (W-AMA) for Deep Neural Networks (DNNs). Considering the Gaussian-like weight distribution, it deploys an accuracy-configurable computing component to improve the computational efficiency. Two techniques for effectively integrating the W-AMA into DNN accelerator are presented: (1) A Cartesian Genetic Programming (CGP) based approximate multiplier is designed and selectable to compute the Least Significant Bit (LSB) for a higher accuracy mode. The Reward–Penalty-Coefficient (RPC) is proposed to achieve the internal-compensation. (2) The Hessian-Aware-Approximation (HAA) method is utilized for hybrid approximate modes cross-layer mapping. Based on the W-AMA, an energy-efficient DNN accelerator is proposed and evaluated on 28 nm technology. It can achieve the energy efficiency of 9.6 TOPS/W, and the computational energy efficiency can be improved by 1.5× compared with the standard units, with an 0.52% accuracy loss on CIFAR-10 using ResNet-18.

Gallium‐Based Liquid–Solid Biphasic Conductors for Soft Electronics

AbstractSoft and stretchable electronics have diverse applications in the fields of compliant bioelectronics, textile‐integrated wearables, novel forms of mechanical sensors, electronics skins, and soft robotics. In recent years, multiple material architectures have been proposed for highly deformable circuits that can undergo large tensile strains without losing electronic functionality. Among them, gallium‐based liquid metals benefit from fluidic deformability, high electrical conductivity, and self‐healing property. However, their deposition and patterning is challenging. Biphasic material architectures are recently proposed as a method to address this problem, by combining advantages of solid‐phase materials and composites, with liquid deformability and self‐healing of liquid phase conductors, thus moving toward scalable fabrication of reliable stretchable circuits. This article reviews recent biphasic conductor architectures that combine gallium‐based liquid‐phase conductors, with solid‐phase particles and polymers, and their application in fabrication of soft electronic systems. In particular, various material combinations for the solid and liquid phases in the biphasic conductor, as well as methods used to print and pattern biphasic conductive compounds, are discussed. Finally, some applications that benefit from biphasic architectures are reviewed.

Infant movement classification through pressure distribution analysis

BackgroundAiming at objective early detection of neuromotor disorders such as cerebral palsy, we propose an innovative non-intrusive approach using a pressure sensing device to classify infant general movements. Here we differentiate typical general movement patterns of the “fidgety period” (fidgety movements) vs. the “pre-fidgety period” (writhing movements).MethodsParticipants (N = 45) were sampled from a typically-developing infant cohort. Multi-modal sensor data, including pressure data from a pressure sensing mat with 1024 sensors, were prospectively recorded for each infant in seven succeeding laboratory sessions in biweekly intervals from 4 to 16 weeks of post-term age. 1776 pressure data snippets, each 5 s long, from the two targeted age periods were taken for movement classification. Each snippet was pre-annotated based on corresponding synchronised video data by human assessors as either fidgety present or absent. Multiple neural network architectures were tested to distinguish the fidgety present vs. fidgety absent classes, including support vector machines, feed-forward networks, convolutional neural networks, and long short-term memory networks.ResultsHere we show that the convolution neural network achieved the highest average classification accuracy (81.4%). By comparing the pros and cons of other methods aiming at automated general movement assessment to the pressure sensing approach, we infer that the proposed approach has a high potential for clinical applications.ConclusionsWe conclude that the pressure sensing approach has great potential for efficient large-scale motion data acquisition and sharing. This will in return enable improvement of the approach that may prove scalable for daily clinical application for evaluating infant neuromotor functions.

Vectorizing and distributing number‐theoretic transform to count Goldbach partitions on Arm‐based supercomputers

SummaryIn this article, we explore the usage of scalable vector extension (SVE) to vectorize number‐theoretic transforms (NTTs). In particular, we show that 64‐bit modular arithmetic operations, including modular multiplication, can be efficiently implemented with SVE instructions. The vectorization of NTT loops and kernels involving 64‐bit modular operations was not possible in previous Arm‐based single instruction multiple data architectures since these architectures lacked crucial instructions to efficiently implement modular multiplication. We test and evaluate our SVE implementation on the A64FX processor in an HPE Apollo 80 system. Furthermore, we implement a distributed NTT for the computation of large‐scale exact integer convolutions. We evaluate this transform on HPE Apollo 70, Cray XC50, HPE Apollo 80, and HPE Cray EX systems, where we demonstrate good scalability to thousands of cores. Finally, we describe how these methods can be utilized to count the number of Goldbach partitions of all even numbers to large limits. We present some preliminary results concerning this problem, in particular a histogram of the number of Goldbach partitions of the even numbers up to 240.

Deployment of the deep learning fusion method to emotional semantic evaluation of natural language

The emotional semantic evaluation of natural language plays a crucial role in sentiment analysis. Deep learning methods have shown great potential in capturing the complex relationships between words and emotions. This paper proposes a deep learning fusion method for deploying emotional semantic evaluation. The technique combines multiple deep learning architectures to capture local and global contextual information, including Bidirectional Gated Recurrent Units (GRU), Long Short-Term Memory (LSTM) networks, and Self-Attention mechanisms. Pretrained GloVe word embedding’s utilized to enhance word representation. A novel fusion layer combines the outputs of individual models; employing self-attention means to assign weights dynamically. This allows the model to weigh the importance of different representations in the final prediction. Benchmark movie review (MR) for sentiment analysis and emotion classification tasks are used to evaluate the proposed method. Experimental results demonstrate superior performance compared to individual deep learning models and traditional feature-based approaches. The proposed fusion method effectively captures the nuances of emotional semantics in natural language, leading to more accurate and nuanced evaluations.